Lead the technical vision and development of scalable ML and AI evaluation systems for Atlassian’s Rovo Chat. Build agentic capabilities, datasets, simulations, automated judges, experimentation platforms, and quality signals. Identify failure modes, ship interventions, and improve customer outcomes while balancing quality, latency, reliability, safety, and cost. Provide cross-organizational technical leadership, mentorship, design guidance, and strategic direction for AI products.
Engineering | Seattle, United States | Remote, Remote
Working at Atlassian
Atlassians can choose where they work - whether in an office, from home, or a combination of the two. That way, Atlassians have more control over supporting their family, personal goals, and other priorities. We can hire people in any country where we have a legal entity.
As a Principal ML Engineer in the Rovo & AI Engineering org, you will contribute to Atlassian's biggest bet and fastest area of growth, Rovo Chat. You will bring the latest bleeding edge innovations to the product and build agent harnesses that deliver a step function improvement in quality, reliability and latency.
You will have the opportunity to lead other engineers to drive involved projects from technical design to launch. You will also collaborate with other teams and internal customers to set expectations, gather input and communicate results. Your role is pivotal, stretching beyond these tasks, ensuring AI's transformative potential is realized across our offerings.
What you'll do
As a Principal Machine Learning Engineer, you'll tackle the open-ended technical problems that determine whether customers trust Rovo with increasingly complex and consequential work. You'll remain deeply hands-on while setting direction across teams: driving advances in models, agents, and evaluation and turning them into measurable improvements for millions of active users.
Your Background
At Atlassian, we strive to design equitable, explainable, and competitive compensation programs. We follow consistent hiring practices and account for each candidate's skills, knowledge, and experience when setting base pay within the range.
Please visit
Working at Atlassian
Atlassians can choose where they work - whether in an office, from home, or a combination of the two. That way, Atlassians have more control over supporting their family, personal goals, and other priorities. We can hire people in any country where we have a legal entity.
As a Principal ML Engineer in the Rovo & AI Engineering org, you will contribute to Atlassian's biggest bet and fastest area of growth, Rovo Chat. You will bring the latest bleeding edge innovations to the product and build agent harnesses that deliver a step function improvement in quality, reliability and latency.
You will have the opportunity to lead other engineers to drive involved projects from technical design to launch. You will also collaborate with other teams and internal customers to set expectations, gather input and communicate results. Your role is pivotal, stretching beyond these tasks, ensuring AI's transformative potential is realized across our offerings.
What you'll do
As a Principal Machine Learning Engineer, you'll tackle the open-ended technical problems that determine whether customers trust Rovo with increasingly complex and consequential work. You'll remain deeply hands-on while setting direction across teams: driving advances in models, agents, and evaluation and turning them into measurable improvements for millions of active users.
- Set the technical vision for how Atlassian measures and improves Rovo Chat, translating qualities such as helpfulness, correctness, judgment, and reliability into rigorous evaluation strategies.
- Architect and build production ML and evaluation systems that operate at scale, including datasets, simulations, automated judges, regression suites, experimentation platforms, and online quality signals.
- Advance the state of Rovo's agentic capabilities across reasoning, planning, tool selection and use, action correctness, retrieval, memory, and long-horizon execution.
- Close the loop between evaluation and product impact: identify systemic failure modes, develop interventions, ship them to production, and establish whether they improved customer outcomes.
- Make high-leverage technical decisions across model choice, prompting, fine-tuning, data strategy, system architecture, and inference; balancing quality with latency, reliability, safety, and cost.
- Lead ambitious, cross-organizational initiatives with ML, engineering, product, analytics, research, and design partners, creating clarity and momentum where neither the problem nor the solution is well defined.
- Establish reusable methods, systems, and quality standards that raise the bar not only for Rovo Chat, but for AI-powered experiences across Atlassian.
- Multiply the impact of other engineers through technical leadership, design reviews, mentorship, and a compelling long-term vision for how we build and evaluate world-class AI products.
Your Background
- A track record of architecting and shipping large-scale ML or AI systems that deliver measurable customer and business impact.
- Deep experience with LLM-powered products, agentic systems, model evaluation, experimentation, and dataset development.
- Strong production engineering skills and the ability to remain hands-on in a modern ML stack.
- Principal-level technical leadership: setting direction, influencing across teams, and leading complex initiatives from ambiguity to production.
- Excellent product and technical judgment, including balancing quality, speed, reliability, safety, latency, and cost.
- The ability to communicate complex ideas clearly and raise the effectiveness of engineers around you through mentorship and technical leadership.
At Atlassian, we strive to design equitable, explainable, and competitive compensation programs. We follow consistent hiring practices and account for each candidate's skills, knowledge, and experience when setting base pay within the range.
Please visit
Atlassian Boston, Massachusetts, USA Office
55 Causeway St, Boston, Massachusetts , United States, 02114
Atlassian Boston, Massachusetts, USA Office
Atlassian believes the future of work is distributed and offers our people the flexibility to help them do what’s important to them. And with few exceptions, we hire people anywhere we have a legal entity as long as they have eligible work rights and sufficient team time zone overlap.
Similar Jobs at Atlassian
Cloud • Information Technology • Productivity • Security • Software • App development • Automation
Lead the design and delivery of knowledge graph and machine learning systems that infer personal work context from connected tools. Build graph inference pipelines, schemas, permission-aware APIs, Rovo Chat integrations, and MCP-compatible CLI experiences. Establish evaluation frameworks, improve retrieval and grounding, provide technical direction across teams, mentor engineers, and ensure responsible, privacy-safe AI systems operate reliably at production scale.
Top Skills:
Amazon NeptuneCypherDglGraph DatabasesGraph Neural NetworksGraphxKnowledge GraphsLlm OrchestrationMcpNeo4JOntology DesignPygRagRetrieval-Augmented GenerationSemantic EmbeddingsSparql
Cloud • Information Technology • Productivity • Security • Software • App development • Automation
Develop and implement advanced machine learning algorithms and models for Atlassian products. Design system and model architectures, conduct experiments and evaluations, collaborate with product, engineering, and analytics teams, and mentor emerging machine learning engineers. The role also contributes to shared AI infrastructure and capabilities supporting search, knowledge discovery, and conversational experiences across the company.
Top Skills:
Ai InfrastructureArtificial IntelligenceData PipelinesMachine LearningMachine Learning AlgorithmsMachine Learning FrameworksModel EvaluationModel Training
Cloud • Information Technology • Productivity • Security • Software • App development • Automation
Principal Machine Learning Engineer responsible for setting technical direction and building production-grade AI systems for Confluence. The role spans model evaluation, retrieval, ranking, prompt and workflow design, experimentation, reliability, and customer-facing AI product development. Responsibilities include making architectural decisions, partnering across engineering and product teams, mentoring engineers, identifying model and product failure modes, and delivering scalable AI experiences across content creation, editing, discovery, recommendations, and multimodal interaction.
Top Skills:
Ai Systems InfrastructureArtificial IntelligenceAutomated EvaluationExperimentationMachine LearningMultimodal AiPrompt EngineeringRankingRetrieval
What you need to know about the Boston Tech Scene
Boston is a powerhouse for technology innovation thanks to world-class research universities like MIT and Harvard and a robust pipeline of venture capital investment. Host to the first telephone call and one of the first general-purpose computers ever put into use, Boston is now a hub for biotechnology, robotics and artificial intelligence — though it’s also home to several B2B software giants. So it’s no surprise that the city consistently ranks among the greatest startup ecosystems in the world.
Key Facts About Boston Tech
- Number of Tech Workers: 269,000; 9.4% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Thermo Fisher Scientific, Toast, Klaviyo, HubSpot, DraftKings
- Key Industries: Artificial intelligence, biotechnology, robotics, software, aerospace
- Funding Landscape: $15.7 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Summit Partners, Volition Capital, Bain Capital Ventures, MassVentures, Highland Capital Partners
- Research Centers and Universities: MIT, Harvard University, Boston College, Tufts University, Boston University, Northeastern University, Smithsonian Astrophysical Observatory, National Bureau of Economic Research, Broad Institute, Lowell Center for Space Science & Technology, National Emerging Infectious Diseases Laboratories

