As a Lead Data Engineer, you'll architect and maintain data pipelines, ensuring their reliability and scalability, while collaborating closely with AI and data teams.
Imagine having an enterprise-grade AppStore at work — one that ensures you can easily search, request, and gain access to any app you need, precisely when you need it. No more long waiting times with outstanding IT requests. Lumos is solving the app and access management challenges for organizations of all sizes through a unified platform. Our fast-growing startup is pioneering the way to untangle the complex web of app and access management by building the critical infrastructure that defines relationships between app, identities and data.
Why Lumos?
- Jump on a Rocketship: Since launching out of stealth mode just over 2 years ago, our team has grown from 20 to ~100 people and our customer base has 10x’ed with companies like GitHub, MongoDB and Major League Baseball!
- Build with Renowned Investor Backing: Andreessen Horowitz (a16z) backed us since the beginning and we've raised over $65m from Scale, Neo, Greg Brockman (President at OpenAI), Phil Venables (CISO at Google), and others.
- Thrive in a Unique Culture: You’ll join an early-stage company where you have actual influence on the trajectory of the company. We deeply care about our people and the philosophy we live by - check out our values here.
Lumos is making it a joy for companies to manage their apps and identities ✨. By integrating usage, spend, compliance, and access data, we provide a level of clarity and insight previously unimaginable. To deliver a best-in-class product, Lumos depends on state-of-the-art data pipelines that power our analytics and AI-driven solutions.
We are seeking a Lead Data Engineer to expand and enhance our existing data infrastructure, built around MySQL, Fivetran, Airbyte, and Snowflake. In this role, you will design and implement production-ready data pipelines with a strong emphasis on reliability, testing, and scalability. Your work will ensure that our AI products and in-product analytics perform flawlessly at scale, driving value for our customers.
✨ Your Responsibilities
- Your mission is to architect, build, and maintain cutting-edge data pipelines that empower our AI products, in-product analytics, and our integration sync systems.
- You will ensure the scalability, reliability, and quality of our analytics data infrastructure, enabling the seamless integration of usage, spend, compliance, and access data to drive business insights and deliver exceptional value to our customers.
- By focusing on testing, automation, and best-in-class engineering practices, you will play a pivotal role in transforming complex data into actionable intelligence, fueling Lumos' growth and innovation.
🙌 What We Value
- Extensive experience designing and implementing medallion architectures (bronze, silver, gold layers) or similar data warehouse paradigms. Skilled in optimizing data pipelines for both batch and real-time processing.
- Extensive experience managing both the OLTP layer (e.g. MySQL) and OLAP layer (e.g. Snowflake).
- Proficiency in deploying data pipelines using CI/CD tools and integrating automated data quality checks, version control, and deployment automation to ensure reliable and repeatable data processes.
- Expertise in advanced SQL, ETL processes, and data transformation techniques. Strong programming skills in Python.
- Demonstrated ability to work closely with AI engineers, data scientists, product engineers, product managers, and other stakeholders to ensure that data pipelines meet the needs of all teams.
💰 Pay Range
- $170,000 - $215,000. Note that this range is a good faith estimate of likely pay for this role; upon hire, the pay may differ due to skill and/or level of experience.
💸 Benefits and Perks:
- 💯 Remote work culture (+/-4 hours Pacific Time)
- ⛑ Medical, Vision, & Dental coverage covered by Lumos
- 🛩 Company and team bonding trips throughout the year fully covered by Lumos
- 💻 Optimal WFH setup to set you up for success
- 🌴 Unlimited PTO, with minimum time off to make sure you are rested and able to be at your best
- 👶🏽 Up to 16 weeks for expecting parents
- 💰 Wellness stipend to keep you awesome and healthy
- 🏦 401k matching plan
Top Skills
Airbyte
Ci/Cd Tools
Fivetran
MySQL
Python
Snowflake
SQL
Similar Jobs
Fintech
The Senior Data Engineer will lead the development of scalable data solutions, mentor engineers, and ensure data quality and compliance while collaborating with various stakeholders.
Top Skills:
GCPNode.jsPythonSQLTerraformTypescript
Healthtech • Software • Analytics • Biotech • Pharmaceutical • Manufacturing
Lead data engineering projects for Market Access at Takeda, ensuring data quality and delivering innovative data solutions using modern cloud technologies.
Top Skills:
AWSDatabricksDeltaEc2GlueIam RolesJavaScriptJSONJupiter NotebooksLambdaParquetPythonRRedshiftS3SagemakerSparkSQLTableau
Healthtech • Software • Analytics • Biotech • Pharmaceutical • Manufacturing
The Lead Data Engineer will develop data strategies and architectures, manage data products, and collaborate with stakeholders to deliver innovative data solutions in the GI BU. Responsibilities include leading projects, maintaining data pipelines, and guiding technical efforts with a focus on cloud-based technologies.
Top Skills:
AWSDatabricksDeltaDjangoEc2GlueIamJSONJupiter NotebooksLambdaMxnetParquetPythonR ShinyRedshiftS3SagemakerSparkSpringSQLTableauTensorFlow
What you need to know about the Boston Tech Scene
Boston is a powerhouse for technology innovation thanks to world-class research universities like MIT and Harvard and a robust pipeline of venture capital investment. Host to the first telephone call and one of the first general-purpose computers ever put into use, Boston is now a hub for biotechnology, robotics and artificial intelligence — though it’s also home to several B2B software giants. So it’s no surprise that the city consistently ranks among the greatest startup ecosystems in the world.
Key Facts About Boston Tech
- Number of Tech Workers: 269,000; 9.4% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Thermo Fisher Scientific, Toast, Klaviyo, HubSpot, DraftKings
- Key Industries: Artificial intelligence, biotechnology, robotics, software, aerospace
- Funding Landscape: $15.7 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Summit Partners, Volition Capital, Bain Capital Ventures, MassVentures, Highland Capital Partners
- Research Centers and Universities: MIT, Harvard University, Boston College, Tufts University, Boston University, Northeastern University, Smithsonian Astrophysical Observatory, National Bureau of Economic Research, Broad Institute, Lowell Center for Space Science & Technology, National Emerging Infectious Diseases Laboratories
