Build and operate secure, scalable AI compute platform services, including backend APIs, asynchronous workflows, Kubernetes controllers, and multi-tenant workload capabilities. The role focuses on distributed training and inference, platform reliability, observability, developer tooling, and infrastructure simplification. Responsibilities include collaborating across engineering teams, improving scalability and developer productivity, and providing technical leadership for production AI platform capabilities.
As a software Engineer on the AI Compute Platform team, you will design and build a secure, reliable, and easy-to-use platform for running AI workloads at scale. You will develop control-plane services, APIs, orchestration systems, and developer tools for distributed training and inference, working closely with compute infrastructure teams, AI engineers, and internal users to improve reliability, scalability, and developer productivity across Arm's AI workloads.
Responsibilities:
Essential Skills and Experience:
Desired Skills and Experience:
In Return:
You will have the opportunity to work on important technical challenges in AI and platform engineering as part of a growing team in Seattle. This position offers a chance to contribute to core platform capabilities. You'll work alongside skilled colleagues from various subject areas and help invent the way new AI services are built and scaled at Arm.
In addition to meaningful technical work, Arm offers an environment that values thoughtful engineering, collaboration, and continuous learning. This opportunity suits someone who enjoys solving complex systems problems, working across teams, and contributing to technology important to the future of AI at Arm.
Additional Information
Please note that a relocation package (including visa sponsorship assistance) is offered for this role, for candidates who need it.
We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of race, religion, color, national origin, sex, gender, gender expression, sexual orientation, age, marital status, veteran status, or disability status. We will ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application or interview process, to perform essential job functions, and to receive other benefits and privileges of employment. Please contact us to request accommodation.
Salary Range:
$209,100-$282,900 per year
We value people as individuals and our dedication is to reward people competitively and equitably for the work they do and the skills and experience they bring to Arm. Salary is only one component of Arm's offering. The total reward package will be shared with candidates during the recruitment and selection process.
Our 10x Mindset
At Arm, we believe progress happens when people are empowered to think bigger and push beyond what seems possible. Our 10x mindset is about curiosity, ambition and creating impact that will be used by millions. We learn fast, prioritise collaboration and turn bold ideas into real technology. We look for people who are inspired by this way of working and want to grow in an environment where bold ideas are welcomed. Discover how the 10x mindset shapes life at Arm: https://careers.arm.com/10x-mindset
Use of AI in Your Application
At Arm, we embrace the benefits AI can bring, including in recruitment. You may use AI to support your preparation, such as improving the clarity of written responses, practicing interview questions, or learning more about our industry. However, your application and interview responses must reflect your own thinking, experience, and abilities. We want to understand your authentic perspective, creativity, and approach to problem solving. As such, you may not use AI to complete assigned tasks as part of any assessments. If you plan to use AI-supported or assistive technologies as part of a reasonable adjustment, please contact [email protected] so we can support you.
Recruitment Adjustments and Accommodations
At Arm, we want to build extraordinary teams. If you need an adjustment or an accommodation during the recruitment process, please email [email protected] . To note, by sending us the requested information, you consent to its use by Arm to arrange for appropriate accommodations. All accommodation or adjustment requests will be treated with confidentiality, and information concerning these requests will only be disclosed as necessary to provide the accommodation. Although this is not an exhaustive list, examples of support include breaks between interviews, having documents read aloud, or office accessibility. Please email us about anything we can do to accommodate you during the recruitment process.
Hybrid Working
Arm's approach to hybrid working is designed to create a working environment that supports both high performance and personal wellbeing. We believe in bringing people together face to face to enable us to work at pace, whilst recognizing the value of flexibility. Within that framework, we empower groups/teams to determine their own hybrid working patterns, depending on the work and the team's needs. Details of what this means for each role will be shared upon application. In some cases, the flexibility we can offer is limited by local legal, regulatory, tax, or other considerations, and where this is the case, we will collaborate with you to find the best solution. Please talk to us to find out more about what this could look like for you.
Equal Opportunities
Arm is an equal opportunity employer, committed to providing an environment of mutual respect where equal opportunities are available to all applicants and colleagues. We are a diverse organization of dedicated and innovative individuals, and don't discriminate on the basis of race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or status as a protected veteran.
Responsibilities:
- Build scalable backend services, APIs, and asynchronous workflows for managing AI workloads.
- Develop Kubernetes controllers that manage workload submission, recovery, and long-running service lifecycles.
- Deliver secure multi-tenant capabilities covering access, secrets, quotas, and auditability.
- Improve platform scalability, observability, reliability, and developer workflows.
- Partner across teams to deliver distributed training, evaluation, and inference capabilities while strengthening engineering quality.
Essential Skills and Experience:
- 8+ years of experience building distributed systems, cloud platforms, or production backend services
- Experience with distributed ML training or inference systems
- Strong programming skills in Go, Python, or similar systems languages
- Practical knowledge of APIs, Kubernetes, containers, asynchronous processing, and relational databases.
- Ability to build clear interfaces and simplify infrastructure for engineers and researchers.
- Clear communication and a collaborative approach to problem-solving.
Desired Skills and Experience:
- Experience with Kubernetes operators, schedulers, resource management, or GitOps or equivalent experience
- Knowledge of RPC systems, PostgreSQL, Kafka, OpenTelemetry, Prometheus and Grafana
- Familiarity with distributed AI platforms, inference serving, or frameworks such as PyTorch, Ray, or vLLM
- Experience delivering developer tools or platform capabilities from early build through production
- Technical leadership, mentoring experience, or an advanced degree or equivalent experience in a related field
In Return:
You will have the opportunity to work on important technical challenges in AI and platform engineering as part of a growing team in Seattle. This position offers a chance to contribute to core platform capabilities. You'll work alongside skilled colleagues from various subject areas and help invent the way new AI services are built and scaled at Arm.
In addition to meaningful technical work, Arm offers an environment that values thoughtful engineering, collaboration, and continuous learning. This opportunity suits someone who enjoys solving complex systems problems, working across teams, and contributing to technology important to the future of AI at Arm.
Additional Information
Please note that a relocation package (including visa sponsorship assistance) is offered for this role, for candidates who need it.
We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of race, religion, color, national origin, sex, gender, gender expression, sexual orientation, age, marital status, veteran status, or disability status. We will ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application or interview process, to perform essential job functions, and to receive other benefits and privileges of employment. Please contact us to request accommodation.
Salary Range:
$209,100-$282,900 per year
We value people as individuals and our dedication is to reward people competitively and equitably for the work they do and the skills and experience they bring to Arm. Salary is only one component of Arm's offering. The total reward package will be shared with candidates during the recruitment and selection process.
Our 10x Mindset
At Arm, we believe progress happens when people are empowered to think bigger and push beyond what seems possible. Our 10x mindset is about curiosity, ambition and creating impact that will be used by millions. We learn fast, prioritise collaboration and turn bold ideas into real technology. We look for people who are inspired by this way of working and want to grow in an environment where bold ideas are welcomed. Discover how the 10x mindset shapes life at Arm: https://careers.arm.com/10x-mindset
Use of AI in Your Application
At Arm, we embrace the benefits AI can bring, including in recruitment. You may use AI to support your preparation, such as improving the clarity of written responses, practicing interview questions, or learning more about our industry. However, your application and interview responses must reflect your own thinking, experience, and abilities. We want to understand your authentic perspective, creativity, and approach to problem solving. As such, you may not use AI to complete assigned tasks as part of any assessments. If you plan to use AI-supported or assistive technologies as part of a reasonable adjustment, please contact [email protected] so we can support you.
Recruitment Adjustments and Accommodations
At Arm, we want to build extraordinary teams. If you need an adjustment or an accommodation during the recruitment process, please email [email protected] . To note, by sending us the requested information, you consent to its use by Arm to arrange for appropriate accommodations. All accommodation or adjustment requests will be treated with confidentiality, and information concerning these requests will only be disclosed as necessary to provide the accommodation. Although this is not an exhaustive list, examples of support include breaks between interviews, having documents read aloud, or office accessibility. Please email us about anything we can do to accommodate you during the recruitment process.
Hybrid Working
Arm's approach to hybrid working is designed to create a working environment that supports both high performance and personal wellbeing. We believe in bringing people together face to face to enable us to work at pace, whilst recognizing the value of flexibility. Within that framework, we empower groups/teams to determine their own hybrid working patterns, depending on the work and the team's needs. Details of what this means for each role will be shared upon application. In some cases, the flexibility we can offer is limited by local legal, regulatory, tax, or other considerations, and where this is the case, we will collaborate with you to find the best solution. Please talk to us to find out more about what this could look like for you.
Equal Opportunities
Arm is an equal opportunity employer, committed to providing an environment of mutual respect where equal opportunities are available to all applicants and colleagues. We are a diverse organization of dedicated and innovative individuals, and don't discriminate on the basis of race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or status as a protected veteran.
Arm Waltham, Massachusetts, USA Office
Arm Boston, MA Office
100 5th Ave, Waltham, MA, United States, 02451
Similar Jobs at Arm
Artificial Intelligence • Internet of Things • Semiconductor
Design and build MLOps infrastructure for production deployment of data, machine-learning models, and software. Develop end-to-end ML pipelines, CI/CD/CT workflows, secure repository strategies, containerized deployments, and automation for Edge AI applications. Collaborate with developers, data scientists, and ML engineers to define tooling, monitor DevOps performance, and improve operational efficiency. Support embedded-system CI/CT, open-source pipelines, and model testing.
Top Skills:
AnsibleAWSAzureCC++DockerElasticsearchGerritGitGithub ActionsGitlab CiGrafanaJenkinsKibanaKubernetesLinuxPackerPrometheusPythonPyTorchTensorFlowTerraform
Artificial Intelligence • Internet of Things • Semiconductor
Leads architecture and hands-on development of distributed AI inference runtime capabilities, including scheduling, batching, KV-cache management, memory allocation, kernel optimization, model execution, and performance benchmarking. Partners with infrastructure, compiler, hardware, cloud, and research teams to improve latency, throughput, reliability, and resource efficiency. Provides technical leadership, conducts reviews, mentors engineers, and establishes performance-engineering practices for production AI workloads.
Top Skills:
Accelerator ProgrammingAi Inference RuntimesC++CompilersDistributed SystemsHigh-Performance NetworkingKernel DevelopmentModel ParallelismOperating SystemsPythonRust
Artificial Intelligence • Internet of Things • Semiconductor
Leads pre-silicon quality engineering for high-speed I/O subsystems including DDR, PCIe, eUSB, and SerDes. Drives DFX metric closure, validates silicon-to-simulation correlation, analyzes eye diagrams, BER, margining, and BIST data, and develops test automation and visualization scripts. Chairs TPIP councils and collaborates with vendors, architects, integration, validation, package, and SIPI teams to ensure quality across upcoming systems-on-chip.
Top Skills:
BerBistClock-Domain ChecksDdrDfxEusbPciePythonScan CoverageSerdesTcl
What you need to know about the Boston Tech Scene
Boston is a powerhouse for technology innovation thanks to world-class research universities like MIT and Harvard and a robust pipeline of venture capital investment. Host to the first telephone call and one of the first general-purpose computers ever put into use, Boston is now a hub for biotechnology, robotics and artificial intelligence — though it’s also home to several B2B software giants. So it’s no surprise that the city consistently ranks among the greatest startup ecosystems in the world.
Key Facts About Boston Tech
- Number of Tech Workers: 269,000; 9.4% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Thermo Fisher Scientific, Toast, Klaviyo, HubSpot, DraftKings
- Key Industries: Artificial intelligence, biotechnology, robotics, software, aerospace
- Funding Landscape: $15.7 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Summit Partners, Volition Capital, Bain Capital Ventures, MassVentures, Highland Capital Partners
- Research Centers and Universities: MIT, Harvard University, Boston College, Tufts University, Boston University, Northeastern University, Smithsonian Astrophysical Observatory, National Bureau of Economic Research, Broad Institute, Lowell Center for Space Science & Technology, National Emerging Infectious Diseases Laboratories
