Build and operate production machine learning training systems on Amazon SageMaker using AWS Trainium. The role focuses on optimizing PyTorch code for Trainium hardware, understanding NeuronCore compilation, diagnosing device-level issues, tuning distributed training for cost and throughput, and delivering production pipelines. The engineer will collaborate directly with client and internal teams in a forward-deployed engineering environment.
Robots & Pencils is an AWS Partner building production AI systems for enterprise clients who need real engineering, not proofs of concept that never ship. We work forward-deployed, embedded directly with client teams, solving the problems that are too new or too specialized for a typical vendor relationship to handle.
The Role
We're looking for an engineer who can operate and train models on Amazon SageMaker running on AWS Trainium, AWS's custom silicon built specifically for large-scale model training. This isn't a role where you call an API and wait. You'll be walking up the stack: understanding what a training request actually looks like at the Trainium hardware and compiler level, then carrying that understanding all the way up through PyTorch training code and into a production SageMaker pipeline.
PyTorch is the backbone of this work. If you know the framework deeply and you're comfortable reasoning about how your code actually behaves on custom accelerator hardware rather than treating it as a black box, this role is built around that skill set specifically.
What You'll Do
- Train and operate models on Amazon SageMaker with AWS Trainium as the underlying compute
- Write and optimize PyTorch training code with a real understanding of how it compiles and executes on Trainium (NeuronCore architecture, compiler behavior, memory and throughput tradeoffs)
- Diagnose training run issues that show up specifically because of the hardware, not just the model, distinguishing a data or code problem from a compiler or device-level one
- Translate a request for "a Trainium job" into an actual working, cost-aware training pipeline, end to end
- Tune distributed training runs for throughput and cost on SageMaker's training infrastructure
- Work directly with client and internal engineering teams to scope and deliver real production training workloads, not experiments that stay in a notebook
What You'll Bring
- Strong, hands-on PyTorch experience, ideally including distributed or multi-device training
- Production experience with Amazon SageMaker for training and/or inference
- Comfort working close to the hardware layer: you understand device-specific compilation and can debug issues that are actually about the accelerator, not just the model
- AWS Trainium or Inferentia (Neuron SDK) experience is a strong plus; if you don't have it yet but have deep PyTorch and a track record of picking up new hardware targets fast, we want to talk to you
- Solid Python fundamentals and comfort operating in a client-facing, production engineering environment
Similar Jobs
Automotive • Hardware • Internet of Things • Mobile • Software • App development • PropTech
Develop and maintain iOS, Android, and web development platforms for the myQ connectivity program. Lead technical planning across scrum teams, translate product requirements into software designs, implement complex features, oversee releases, improve development processes, and enforce quality standards. The role mentors engineers, leads offshore teams, collaborates with product, architecture, DevOps, firmware, infrastructure, and QA teams, and contributes to architectural decisions, technical documentation, interviews, and continuous improvement.
Top Skills:
AndroidCi/CdGitiOSJSONKotlinMvcMvvmRestScrumSwift
Cloud • Fintech • Information Technology • Machine Learning • Software
Lead large-scale software engineering efforts focused on modernizing monolithic systems into distributed, scalable architectures. Make major code design decisions, develop technical solutions, improve CI/CD tooling and TDD practices, and maintain reliable systems using C#, .NET, React, AWS, and Kubernetes. Mentor engineers, contribute to engineering standards, participate in agile delivery and code reviews, and collaborate with product, design, and international engineering teams.
Top Skills:
.NetAutomated TestingAWSC#Ci/CdDistributed SystemsKubernetesReactTdd
Artificial Intelligence • Cloud • HR Tech • Information Technology • Productivity • Software • Automation
Lead the vision, strategy, and development of ServiceNow’s enterprise search, RAG, question-answering, and conversational AI capabilities. Drive measurable customer and business outcomes, guide engineering and ML partners, incorporate user and market insights, define performance metrics, expand adoption across the platform, and manage and mentor a team of at least four Product Managers.
Top Skills:
Artificial IntelligenceCoveoData IndexingElasticsearchEnterprise SearchGleanMachine LearningMoveworksRelevance RankingRetrieval-Augmented Generation (Rag)
What you need to know about the Boston Tech Scene
Boston is a powerhouse for technology innovation thanks to world-class research universities like MIT and Harvard and a robust pipeline of venture capital investment. Host to the first telephone call and one of the first general-purpose computers ever put into use, Boston is now a hub for biotechnology, robotics and artificial intelligence — though it’s also home to several B2B software giants. So it’s no surprise that the city consistently ranks among the greatest startup ecosystems in the world.
Key Facts About Boston Tech
- Number of Tech Workers: 269,000; 9.4% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Thermo Fisher Scientific, Toast, Klaviyo, HubSpot, DraftKings
- Key Industries: Artificial intelligence, biotechnology, robotics, software, aerospace
- Funding Landscape: $15.7 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Summit Partners, Volition Capital, Bain Capital Ventures, MassVentures, Highland Capital Partners
- Research Centers and Universities: MIT, Harvard University, Boston College, Tufts University, Boston University, Northeastern University, Smithsonian Astrophysical Observatory, National Bureau of Economic Research, Broad Institute, Lowell Center for Space Science & Technology, National Emerging Infectious Diseases Laboratories
