Do you enjoy shaping frontier AI technologies such as AI agents and agent evaluation, one of the fastest-growing areas in AI today? Join our team to help build the data-driven intelligence behind Microsoft 365 Copilot, applying advanced analytics and machine learning to improve the quality, intelligence, and user experience of AI-powered productivity solutions.
As an Applied Scientist, you'll work at the intersection of large language models (LLMs), information retrieval, search, machine learning, and AI evaluation, partnering closely with software engineers, researchers, and product managers to develop data-driven approaches that measure, analyze, and improve AI systems. You'll analyze large-scale datasets, design rigorous experiments, uncover actionable insights, and help bring cutting-edge AI innovations from idea to production.
Our team embraces an AI-first engineering culture, leveraging AI throughout the development lifecycle to accelerate innovation while maintaining the highest standards of security, quality, and trust.
Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.
Responsibilities
Develop metrics and evaluation methodologies to measure the quality, accuracy, and usability of AI systems.
Design and analyze online and offline experiments to quantify product improvements and identify opportunities for optimization.
Build scalable data pipelines, dashboards, and analytical tools that enable continuous quality monitoring.
Apply statistical analysis, machine learning, and data mining techniques to extract insights from large-scale datasets.
Partner with engineering, research, and product teams to prioritize investments using data-driven recommendations.
Investigate quality regressions, identify root causes, and propose measurable improvements.
Shape best practices for AI evaluation, experimentation, and responsible AI measurement across multiple products.
Communicate technical findings clearly to both engineering and business stakeholders.
Qualifications
Required Qualifications:
- Bachelor's Degree in Statistics, Econometrics, Computer Science, Electrical or Computer Engineering, or related field AND 4+ years related experience (e.g., statistics predictive analytics, research)
- OR Master's Degree in Statistics, Econometrics, Computer Science, Electrical or Computer Engineering, or related field AND 3+ years related experience (e.g., statistics, predictive analytics, research)
- OR Doctorate in Statistics, Econometrics, Computer Science, Electrical or Computer Engineering, or related field AND 1+ year(s) related experience (e.g., statistics, predictive analytics, research)
- OR equivalent experience.
- Master's Degree in Statistics, Econometrics, Computer Science, Electrical or Computer Engineering, or related field AND 6+ years related experience (e.g., statistics, predictive analytics, research)
- OR Doctorate in Statistics, Econometrics, Computer Science, Electrical or Computer Engineering, or related field AND 3+ years related experience (e.g., statistics, predictive analytics, research)
- OR equivalent experience.
- 3+ years experience creating publications (e.g., patents, libraries, peer-reviewed academic papers).
- Experience presenting at conferences or other events in the outside research/industry community as an invited speaker.
- 3+ years experience conducting research as part of a research program (in academic or industry settings).
- 1+ year(s) experience developing and deploying live production systems, as part of a product team.
- 1+ year(s) experience developing and deploying products or systems at multiple points in the product cycle from ideation to shipping.
Applied Sciences IC4 - The typical base pay range for this role across the U.S. is USD $119,800 - $234,700 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $160,200 - $261,000 per year.
Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:
https://careers.microsoft.com/us/en/us-corporate-pay
This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.
Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.
Microsoft Cambridge, Massachusetts, USA Office
255 Main Street, Cambridge, MA 02142, Cambridge, United States, 02142
Similar Jobs
.png){kind=avatar}
What you need to know about the Boston Tech Scene
Key Facts About Boston Tech
- Number of Tech Workers: 269,000; 9.4% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Thermo Fisher Scientific, Toast, Klaviyo, HubSpot, DraftKings
- Key Industries: Artificial intelligence, biotechnology, robotics, software, aerospace
- Funding Landscape: $15.7 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Summit Partners, Volition Capital, Bain Capital Ventures, MassVentures, Highland Capital Partners
- Research Centers and Universities: MIT, Harvard University, Boston College, Tufts University, Boston University, Northeastern University, Smithsonian Astrophysical Observatory, National Bureau of Economic Research, Broad Institute, Lowell Center for Space Science & Technology, National Emerging Infectious Diseases Laboratories
