Upstart logo, people working, and tagline "Build the future, join the team"
Upstart Logo

Upstart

Senior Software Engineer, LLM Platform

Posted Yesterday
Easy Apply
Remote
2 Locations
164K-226K Annually
Senior level
Easy Apply
Remote
2 Locations
164K-226K Annually
Senior level
As a Senior Software Engineer on the LLM Platform team, you'll design and build infrastructure for AI integration, ensuring performance, security, and compliance.
The summary above was generated by AI

About Upstart

Upstart is the leading AI lending marketplace partnering with banks and credit unions to expand access to affordable credit. By leveraging Upstart's AI marketplace, Upstart-powered banks and credit unions can have higher approval rates and lower loss rates across races, ages, and genders, while simultaneously delivering the exceptional digital-first lending experience their customers demand. More than 80% of borrowers are approved instantly, with zero documentation to upload.

Upstart is a digital-first company, which means that most Upstarters live and work anywhere in the United States. However, we also have offices in San Mateo, California; Columbus, Ohio; and Austin, Texas.

Most Upstarters join us because they connect with our mission of enabling access to effortless credit based on true risk. If you are energized by the impact you can make at Upstart, we’d love to hear from you!

The Team

Upstart’s new LLM Platform team is building foundational infrastructure that democratizes access to generative AI for every product and engineering team across the company.

As a Senior Software Engineer on the LLM  Platform team, you will help design and build a unified interface for securely interacting with large language models - abstracting away the complexity of model selection, prompt orchestration, evaluation, and scaling. You'll help build a centralized GenAI layer that enables engineers to use best-in-class LLMs through clean APIs, configurable pipelines, and intuitive tooling.

Reporting to our Senior Director of Engineering, the team operates with broad scope and high visibility, partnering across Machine Learning, Product, and Compliance to ensure every model integration is performant, secure, and compliant. Whether it's embedding LLMs in user-facing products or optimizing backend workflows, this team makes it seamless for Upstart Engineering to adopt and scale GenAI company-wide.


How you’ll make an impact:

  • Build the LLM Platform powering generative AI use cases across all of Upstart Engineering.
  • Design and implement reliable, observable infrastructure for model inference, prompt orchestration, and data workflows.
  • Drive cross-functional adoption of AI tooling by delivering reusable components and intuitive interfaces.Increase developer productivity through automation, metrics dashboards, and streamlined GenAI integrations.
  • Set the technical roadmap for platform capabilities, balancing innovation with reliability and compliance.
  • Partner with ML researchers, product engineers, and design to bridge experimentation and production.
  • Ensure the platform adheres to emerging standards of security, fairness, and explainability for LLM systems.


What we’re looking for: 

  • Minimum requirements:
    • 6+ years of experience in object-oriented software engineering, with a strong systems engineering background
    • Proficiency in backend development, microservices architecture, and infrastructure tools like Kubernetes, Docker, and Terraform
    • Hands-on experience building ML platforms or infrastructure supporting LLMs or inference systems
    • Familiarity with observability tools (e.g., Prometheus, Grafana, Datadog) and data processing pipelines
    • Strong written and verbal communication skills and a collaborative approach to cross-functional work
  • Preferred qualifications:
    • Full Stack development skills, ideally with Kotlin, Spring, JS/TS (React) or similar
    • Experience with LLM toolchains such as LangChain, LlamaIndex, or OpenAI APIs
    • Deep understanding of model inference optimization (e.g., quantization, ONNX, streaming)
    • Exposure to retrieval-augmented generation (RAG), vector databases (e.g., FAISS, Pinecone), or prompt engineering
    • Experience operationalizing LLMs in production, including latency management and prompt versioning
    • Ability to influence platform direction and drive alignment across multiple engineering teams
    • Awareness of LLM-specific risk domains, including hallucinations, fairness, and data privacy

Position Location - This role is available in the following locations: Remote, San Mateo, Columbus, Austin 

Time Zone Requirements - This team operates across all U.S. time zones.

Travel Requirements - This team has regular on-site collaboration sessions. These occur 3 days per quarter at an Upstart office. If you need to travel to make these meetups, Upstart will cover all travel related expenses.


What you'll love: 

  • Competitive Compensation (base + bonus & equity)
  • Comprehensive medical, dental, and vision coverage with Health Savings Account contributions from Upstart 
  • 401(k) with 100% company match up to $4,500 and immediate vesting and after-tax savings
  • Employee Stock Purchase Plan (ESPP)
  • Life and disability insurance
  • Generous holiday, vacation, sick and safety leave  
  • Supportive parental, family care, and military leave programs
  • Annual wellness, technology & ergonomic reimbursement programs
  • Social activities including team events and onsites, all-company updates, employee resource groups (ERGs), and other interest groups such as book clubs, fitness, investing, and volunteering
  • Catered lunches + snacks & drinks when working in offices

 

#LI-REMOTE

#LI-MidSenior

At Upstart, your base pay is one part of your total compensation package.  The anticipated base salary for this position is expected to be within the below range. Your actual base pay will depend on your geographic location–with our “digital first” philosophy, Upstart uses compensation regions that vary depending on location. Individual pay is also determined by job-related skills, experience, and relevant education or training. Your recruiter can share more about the specific salary range for your preferred location during the hiring process.

In addition, Upstart provides employees with target bonuses, equity compensation, and generous benefits packages (including medical, dental, vision, and 401k).

United States | Remote - Anticipated Base Salary Range
$163,600$226,400 USD

Upstart is a proud Equal Opportunity Employer. We are dedicated to ensuring that underrepresented classes receive better access to affordable credit, and are just as committed to embracing diversity and inclusion in our hiring practices. We celebrate all cultures, backgrounds, perspectives, and experiences, and know that we can only become better together. 

If you require reasonable accommodation in completing an application, interviewing, completing any pre-employment testing, or otherwise participating in the employee selection process, please email [email protected]

https://www.upstart.com/candidate_privacy_policy

Top Skills

Datadog
Docker
Grafana
Kotlin
Kubernetes
Prometheus
Python
React
Spring
Terraform

Similar Jobs at Upstart

3 Hours Ago
Easy Apply
Remote
2 Locations
Easy Apply
157K-217K Annually
Senior level
157K-217K Annually
Senior level
Artificial Intelligence • Fintech • Machine Learning • Social Impact • Software
The Staff Analyst will drive insights and data infrastructure for the Personal Loans product, collaborating with cross-functional teams to set strategy and optimize product performance.
Top Skills: LookerModePythonRSQLTableau
3 Hours Ago
Easy Apply
Remote
2 Locations
Easy Apply
142K-226K Annually
Senior level
142K-226K Annually
Senior level
Artificial Intelligence • Fintech • Machine Learning • Social Impact • Software
The Senior Software Engineer will build and lead web features for Upstart's Small Dollar Loans, mentor engineers, and collaborate on strategy.
Top Skills: C#JavaKotlinNext.JsReactRubyRuby On RailsSpring FrameworkVercel
3 Hours Ago
Easy Apply
Remote
2 Locations
Easy Apply
164K-226K Annually
Senior level
164K-226K Annually
Senior level
Artificial Intelligence • Fintech • Machine Learning • Social Impact • Software
The Senior Analytics Engineer designs and develops data models for analytical stakeholders, ensuring optimal data governance, quality, and orchestration, while guiding teams on best practices for analytics.
Top Skills: AirflowAWSAzureBigQueryBusiness Intelligence Visualization ToolsDatabricksDbtEtl OptimizationGCPLookerPythonRedshiftSQL

What you need to know about the Boston Tech Scene

Boston is a powerhouse for technology innovation thanks to world-class research universities like MIT and Harvard and a robust pipeline of venture capital investment. Host to the first telephone call and one of the first general-purpose computers ever put into use, Boston is now a hub for biotechnology, robotics and artificial intelligence — though it’s also home to several B2B software giants. So it’s no surprise that the city consistently ranks among the greatest startup ecosystems in the world.

Key Facts About Boston Tech

  • Number of Tech Workers: 269,000; 9.4% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Thermo Fisher Scientific, Toast, Klaviyo, HubSpot, DraftKings
  • Key Industries: Artificial intelligence, biotechnology, robotics, software, aerospace
  • Funding Landscape: $15.7 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Summit Partners, Volition Capital, Bain Capital Ventures, MassVentures, Highland Capital Partners
  • Research Centers and Universities: MIT, Harvard University, Boston College, Tufts University, Boston University, Northeastern University, Smithsonian Astrophysical Observatory, National Bureau of Economic Research, Broad Institute, Lowell Center for Space Science & Technology, National Emerging Infectious Diseases Laboratories

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account