Own and evolve end-to-end data pipeline architecture across ingestion, transformation, modeling, and serving. Lead platform improvements for cost efficiency, reliability, access control, observability, orchestration, CI/CD, and developer experience. Drive complex cross-team initiatives, resolve production issues, influence architectural decisions, mentor engineers, and build automation incorporating AI into data engineering workflows.
About the Role
Here's what you'll do as part of the team
Here are the skills and experience you'll need to be successful
Here's the Pay Range:
As Apartment List's data platform grows across more domains, more pipelines, and more stakeholders, the architectural decisions made today determine how much technical debt we're paying down in a year. We're looking for a Senior Data Engineer II (IC4) to own that architecture; end-to-end pipeline design, platform investment tradeoffs, and the technical judgment calls that keep the system reliable as it scales.
This is not a design-from-the-whiteboard role. You'll still be hands-on-keyboard: writing pipelines, debugging production issues, and shipping code alongside the team. What sets this role apart is scope; you'll make architectural decisions independently, influence how Analytics Engineering, Data Science, and Engineering partners build on the platform, and be the person other data engineers come to when a design decision needs a second opinion. You'll work closely with Analytics Engineering, Data Science, and Engineering partners to shape how the data platform evolves.
- Own and evolve data pipeline architecture across core domains — ingestion, transformation, modeling, and serving — making project-level architectural decisions independently and evaluating tradeoffs between freshness, cost, scalability, and simplicity.
- Lead the design and implementation of platform-level improvements: warehouse cost management, compute efficiency, and access control architecture, treating reliability, observability, and cost efficiency as core design constraints rather than afterthoughts.
- Identify and lead technical initiatives that improve the platform's long-term health — proactively surfacing investments (orchestration, CI/CD, data access, developer experience) before they become blockers, and making the case for them.
- Drive large, technically complex projects or multiple concurrent medium-sized initiatives that span teams, taking responsibility for outcomes rather than just execution.
- Lead monitoring and testing strategy for your domain: proactively close observability gaps across the org, build alerting ahead of failures, and serve as the go-to engineer for the hardest production issues.
- Influence technical decisions and architectural direction beyond the Data & Analytics team, partnering directly with EPD stakeholders on infrastructure decisions that affect their roadmaps.
- Actively mentor other data engineers and analytics engineers, reviewing architectural and modeling decisions, and operate as a technical peer to senior engineers across teams.
- Integrate AI meaningfully into data engineering workflows — building tooling and automation that creates leverage for the whole team, not just individual output, and coaching others on effective, validated use.
Must-haves:
- 7+ years of data engineering experience, including a demonstrated track record of owning end-to-end pipeline architecture, not just implementing to spec.
- Deep experience designing orchestration workflows in Apache Airflow, including making architectural tradeoffs across ingestion, transformation, modeling, and serving layers.
- Experience working with containerized data infrastructure in production, including deploying services, diagnosing operational issues, and contributing to platform reliability and scalability.
- Demonstrated ability to evaluate and communicate architectural tradeoffs (freshness vs. cost, scalability vs. simplicity) to both technical and non-technical stakeholders.
- Experience building or significantly improving CI/CD practices for data pipelines, including automated testing, validation, and deployment.
- A track record of leading incident response and monitoring strategy for a domain, including building alerting and observability ahead of failures rather than reacting to them.
- Experience influencing technical decisions across multiple teams or functions, including partnering with engineering, product, or data science stakeholders outside your immediate team.
- Experience mentoring other data engineers, including reviewing architectural and modeling decisions.
Nice-to-haves:
- Experience with Kubernetes-based data infrastructure
- Experience leading a legacy ETL-to-modern-orchestration migration end-to-end, not just contributing to one.
- Familiarity with observability and monitoring tooling such as Datadog at a platform-wide scale.
- Experience building internal tooling or automation (including AI-assisted) that other engineers rely on.
At Apartment List, we carefully consider a variety of factors to determine compensation for each position, including the role, level, and work. The US Total Target Compensation (TTC) for this position is:
- Zone 1: $210,000 - $255,000 TTC (including $189,000 - $229,000 base salary) + equity
- Zone 2: $194,000 - $236,000 TTC (including $175,000 - $212,000 base salary) + equity
- Zone 3: $179,000 - $217,000 TTC (including $161,000 - $195,000 base salary) + equity
This reflects the compensation target for new hire salaries for the position across all US locations. Please note, the compensation details provided do not include benefits and perks that we offer.
We also rely on market indicators along with considering your work location, job related skills, experience and relevant education and training, to determine compensation that is fair and competitive for you. Apartment List will consider paying compensation near the higher of the range in exceptional circumstances, where candidates have the experience, credentials or expertise that would warrant such consideration. It is always our goal to hire exceptional talent and we would be happy to share more about compensation during the hiring process.
Similar Jobs
Artificial Intelligence • Cloud • Computer Vision • Hardware • Internet of Things • Software
Architect and own marketing data infrastructure, including Databricks pipelines, Gold Layer tables, semantic models, and documentation for self-service analytics. Lead cross-functional data initiatives, ensure reliability and quality, mentor engineers, and build production LLM and agent systems. Translate ambiguous business needs into scalable solutions, automate marketing and go-to-market workflows with AI, and deliver high-quality Python and SQL while overseeing AI-generated code.
Top Skills:
Ai AgentsBigQueryClaude CodeCursorDatabricksDbtLlmsPythonSalesforceSnowflakeSQLStreamlit
Artificial Intelligence • Cloud • Computer Vision • Hardware • Internet of Things • Software
Architect and own marketing data infrastructure, including Databricks pipelines, Gold Layer tables, semantic models, and ETL/ELT workflows. Build trusted data products for analytics, automation, and AI agents; translate ambiguous business needs into scalable solutions; lead cross-team initiatives; ensure reliability and data quality; mentor engineers and analysts; and develop, evaluate, and monitor production LLM and agent systems using Python and SQL.
Top Skills:
Ai AgentsBigQueryClaude CodeCursorDatabricksDbtEtl/EltLlmsPythonSalesforceSnowflakeSQLStreamlit
Artificial Intelligence • Cloud • Computer Vision • Hardware • Internet of Things • Software
Architect and own marketing data infrastructure, including Databricks pipelines, Gold Layer tables, semantic models, and trusted analytics foundations. Lead cross-functional data initiatives, translate ambiguous business needs into scalable solutions, ensure data quality and reliability, and mentor engineers and analysts. Build, evaluate, and monitor production LLM and agent systems, automate marketing workflows with AI, and deliver high-quality Python and SQL solutions supporting Marketing, Sales, and R&D.
Top Skills:
Ai AgentsBigQueryClaude CodeCursorData WarehousesDatabricksDbtEtl/EltLakehousesLlmsPythonSalesforceSemantic LayersSnowflakeSQLStreamlit
What you need to know about the Boston Tech Scene
Boston is a powerhouse for technology innovation thanks to world-class research universities like MIT and Harvard and a robust pipeline of venture capital investment. Host to the first telephone call and one of the first general-purpose computers ever put into use, Boston is now a hub for biotechnology, robotics and artificial intelligence — though it’s also home to several B2B software giants. So it’s no surprise that the city consistently ranks among the greatest startup ecosystems in the world.
Key Facts About Boston Tech
- Number of Tech Workers: 269,000; 9.4% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Thermo Fisher Scientific, Toast, Klaviyo, HubSpot, DraftKings
- Key Industries: Artificial intelligence, biotechnology, robotics, software, aerospace
- Funding Landscape: $15.7 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Summit Partners, Volition Capital, Bain Capital Ventures, MassVentures, Highland Capital Partners
- Research Centers and Universities: MIT, Harvard University, Boston College, Tufts University, Boston University, Northeastern University, Smithsonian Astrophysical Observatory, National Bureau of Economic Research, Broad Institute, Lowell Center for Space Science & Technology, National Emerging Infectious Diseases Laboratories
