As a Senior Software Engineer, you'll build and scale data pipelines for machine learning, support data scientists, and ensure platform reliability.
Best Egg is the leading financial confidence platform that provides flexible solutions to real-life challenges for people with limited savings. We leverage real-time customer insights and data science to connect more people with the right products for their financial needs. Our offers include a growing suite of products such as personal loans, a credit card, and flexible rent, which are complemented by a suite of financial health tools to help customers make smart financial decisions and stay on track, so they can be money confident no matter what life throws at them.
Our culture and values inspire our employees and customers to embrace Best Egg. We are committed to championing a culture of inclusiveness and diversity of thought, and we focus on providing a safe, flexible, and collaborative work environment. Our employees are encouraged to engage in creative problem solving, and we promote opportunities for growth and enrichment across the organization.
If you are inspired by inspiring others, Best Egg is the place for you.
As a member of the ML Feature Platform Engineering team, you will play a key role in building and scaling the core data and feature engineering pipelines that power machine learning and AI at Best Egg. Our platform enables data scientists to quickly prototype, train, and deploy models with consistent, reliable, and high-quality features across batch, streaming, and real-time contexts. The platform leverages a modern tech stack including Python, SQL, Polars, Narwhals, Snowflake, AWS, Kubernetes, and FastAPI. Your work will directly impact the productivity of data scientists, the robustness of ML models, and the ability to innovate with new features in production.
Key Responsibilities
- Build Robust Data Pipelines: Design and implement SQL- and Python-based pipelines (using dataframes such as Polars/Narwhals) that support both backfills for training and low-latency, real-time serving.
- Feature Engineering: Collaborate with data scientists to design, build, and maintain new features from complex time series data sources, ensuring they are reusable, well-documented, and consistent across environments.
- Data Debugging & Support: Help data scientists troubleshoot complex SQL queries, debug feature outputs, and optimize queries for performance.
- Platform Reliability: Quickly diagnose and resolve platform issues spanning Python services (FastAPI), Snowflake queries, Kubernetes services, and real-time pipelines.
- Developer Experience: Deliver high-quality abstractions, tools, and libraries that simplify feature development and improve data scientist workflows.
- Performance & Scalability: Monitor, profile, and optimize data pipelines and feature services for throughput, latency, and cost efficiency.
What You'll Need to Succeed
- Professional Experience: 5+ years in data engineering, backend engineering, ML engineering, or related software development roles with a proven track record of building and maintaining large-scale data systems.
- Strong Data Engineering Expertise: Advanced SQL skills, experience with Python dataframes (e.g., Polars, pandas, Narwhals), and a deep understanding of data modeling and feature engineering best practices.
- Hands-On Experience with Cloud Warehouses: Significant experience with Snowflake, BigQuery, Redshift, or Databricks in production settings.
- Pipeline Development Skills: Ability to design Python data pipelines (DAGs) that work seamlessly across batch and real-time contexts, including concepts like incremental processing and backfills.
- Backend Engineering Experience (Secondary but Valuable): Solid Python development skills (concurrency, async, API design) and experience with FastAPI (or similar) for building data services.
- Debugging & Troubleshooting: Comfort in quickly investigating complex issues across SQL, Python, and infrastructure layers.
- DevOps & Deployment Knowledge: Working experience with Docker, Kubernetes, CI/CD workflows, and infrastructure-as-code (Terraform/CloudFormation).
- Collaboration & Mentorship: Ability to partner effectively with data scientists, analysts, and engineers while promoting best practices in feature engineering and data platform design.
Best Egg celebrates diversity and equal opportunity. We are committed to building a team that represents a variety of backgrounds, perspectives, and skills. The more inclusive we are, the better we will grow.
Employee Benefits
Best Egg offers many additional benefits for our employees, including (but not limited to):
· Pre-tax and post-tax retirement savings plans with a competitive company matching
program
· Generous paid time-off plans including vacation, personal/sick time, paid short--
term and long-term disability leaves, paid parental leave, and paid company
holidays
· Multiple health care plans to choose from, including dental and vision options
· Flexible Spending Plans for Health Care, Dependent Care, and Health
Reimbursement Accounts
· Company-paid benefits such as life insurance, wellness platforms, employee
assistance programs, and Health Advocate programs
· Other great discounted benefits include identity theft protection, pet insurance,
fitness center reimbursements, and many more!
Top Skills
AWS
Fastapi
Kubernetes
Narwhals
Polars
Python
Snowflake
SQL
Similar Jobs
Artificial Intelligence • Automotive • Machine Learning • Transportation
The Senior Software Engineer will own large-scale data mining workflows, improve ML products through high-quality datasets, and analyze model performance across driving scenarios.
Top Skills:
AthenaAWSBeamBigQueryEksEmrHadoopKafkaMongoDBMySQLOpensearchPostgresPythonRdsRedshiftS3SparkSQL
Transportation
As a Senior Software Engineer, you will create data pipelines for training and evaluating Waabi's autonomous platform, ensuring quality code and collaborating with a multidisciplinary team.
Top Skills:
Cloud Job OrchestrationData Processing PipelinesDistributed ComputingETLMachine LearningPython
Artificial Intelligence • Automotive • Machine Learning • Transportation
Responsible for building scalable microservices and data pipelines, mentoring junior engineers, and ensuring data quality for ML training.
Top Skills:
AirflowApi GatewayArgoAWSAws Step FunctionsC++DockerDynamoDBEksJavaLambdaOpensearchPythonRedshiftS3
What you need to know about the Boston Tech Scene
Boston is a powerhouse for technology innovation thanks to world-class research universities like MIT and Harvard and a robust pipeline of venture capital investment. Host to the first telephone call and one of the first general-purpose computers ever put into use, Boston is now a hub for biotechnology, robotics and artificial intelligence — though it’s also home to several B2B software giants. So it’s no surprise that the city consistently ranks among the greatest startup ecosystems in the world.
Key Facts About Boston Tech
- Number of Tech Workers: 269,000; 9.4% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Thermo Fisher Scientific, Toast, Klaviyo, HubSpot, DraftKings
- Key Industries: Artificial intelligence, biotechnology, robotics, software, aerospace
- Funding Landscape: $15.7 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Summit Partners, Volition Capital, Bain Capital Ventures, MassVentures, Highland Capital Partners
- Research Centers and Universities: MIT, Harvard University, Boston College, Tufts University, Boston University, Northeastern University, Smithsonian Astrophysical Observatory, National Bureau of Economic Research, Broad Institute, Lowell Center for Space Science & Technology, National Emerging Infectious Diseases Laboratories
