The Senior Data Engineer will architect and optimize data platforms, lead data pipeline design, enforce governance, and mentor junior engineers while ensuring compliance and performance.
As a Senior Data Engineer, you will architect, build, and optimize scalable data platforms that empower our clients with unified, actionable insights. You will lead the design of robust data pipelines, enforce best practices in data governance and security, and mentor junior engineers. Your expertise will bridge technical execution and strategic vision, ensuring high-performance, secure, and scalable data solutions that drive business impact.
Technical Leadership & Strategy:
- Architect and implement enterprise-grade data platforms, including data lakes, warehouses, and real-time pipelines.
- Define and enforce data engineering standards, including ETL/ELT frameworks, data modeling, and quality controls.
- Lead performance optimization of databases, queries, and pipelines (e.g., partitioning, indexing, cost-efficient cloud solutions).
- Stay updated with emerging trends, evaluating and integrating new technologies.
Data Architecture & Infrastructure:
- Design and implement scalable data architecture that ensures high availability, security, and performance.
- Define best practices for data modeling, storage, retrieval, and governance.
- Optimize database structures, schemas, indexing, and partitioning for efficient query performance.
- Implement CI/CD for data pipelines and automate infrastructure using Infrastructure-as-Code (IaC) tools.
Data Pipelines & ETL Development:
- Architect and develop end-to-end ETL/ELT pipelines for structured and unstructured data ingestion, processing, and transformation.
- Automate and optimize data workflows using tools like Apache Spark, Kafka, Airflow, and dbt.
- Monitor, troubleshoot, and optimize data pipeline performance.
Data Governance, Security & Compliance:
- Implement data governance policies, including metadata management, lineage tracking, and role-based access control (RBAC).
- Ensure compliance with data privacy regulations (e.g., GDPR, CCPA) and enforce security best practices.
- Develop frameworks for data quality validation, anomaly detection, and integrity checks.
Business & Stakeholder Engagement:
- Collaborate with stakeholders (clients, PMs) to translate business needs into technical specifications and scalable architectures.
- Develop interactive dashboards and reporting solutions to provide actionable insights.
Innovation & Mentorship:
- Lead technical discussions, workshops, and knowledge-sharing sessions within the team.
- Mentor junior engineers, conduct code reviews, and set best practices.
- Drive automation of data workflows, including CI/CD for pipelines and Infrastructure-as-Code (IaC) solutions.
Technical Skills & Experience:
- 7+ years of experience in data engineering with a proven track record of building scalable data solutions.
- Expertise in SQL and relational database management systems (MySQL, PostgreSQL, Oracle).
- Proficiency in Python (or similar languages) for data processing and automation.
- Deep understanding of cloud platforms (AWS, GCP, Azure) and cloud-native data services:AWS (Redshift, Glue, S3)Azure (Synapse, Data Factory)GCP (BigQuery, Dataflow)
- Hands-on experience with big data tools:Apache Spark, Kafka, Airflow, dbt, Snowflake, Databricks
- Expertise in data modeling.
- Strong knowledge of data security, governance, and compliance frameworks.
Leadership & Soft Skills:
- Proven track record of leading data projects, including migrations, warehousing, and real-time analytics.
- Ability to lead cross-functional teams and manage client expectations.
- Strong problem-solving skills for debugging distributed systems and resolving data anomalies.
- Experience with Agile/Scrum methodologies, including tools like Jira, Git.
- Excellent communication and stakeholder management skills.
Nice-to-Have (Bonus Skills):
- Certifications in cloud/data technologies (e.g., AWS Certified Data Analytics, Google Cloud Data Engineer).
- Experience with ML pipelines (e.g., feature stores, model deployment).
- Contributions to open-source data projects.
Applicant Eligibility: Please note, candidates who are eligible to work in the US without visa sponsorship are eligible to apply. We are not accepting applicants from recruiters or staffing agencies
Top Skills
Airflow
Spark
AWS
Azure
Databricks
Dbt
GCP
Kafka
Python
Snowflake
SQL
Similar Jobs
AdTech • Cloud • Digital Media • Marketing Tech • Analytics • Consulting
Lead design and delivery of data solutions for clients using Adobe Experience Platform. Architect data workflows, mentor junior engineers, and optimize data models.
Top Skills:
Adobe AnalyticsAdobe CampaignAdobe Data CollectionAdobe Experience PlatformAdobe TargetAirflowAWSAzureDbtFivetranGCPJavaScriptPythonReal-Time CdpSnowflakeSQL
.png)
Cloud • Insurance • Payments • Software • Business Intelligence • App development • Big Data Analytics
As a Senior Data Engineer, you will enhance data solutions, manage data models, and collaborate with teams to optimize data architecture and accessibility for insurance business.
Top Skills:
BigQueryGoogle Cloud PlatformPythonSQL
AdTech • Cloud • Digital Media • Information Technology • News + Entertainment • App development
As a Senior Data Engineer, you'll design prototypes, integrate technologies, automate tasks, and support data governance while collaborating with various stakeholders.
Top Skills:
AWSAzureDatabricksGCPHugging FaceOpen Ai ApiPower BIPythonPyTorchSnowflakeSQLTableau
What you need to know about the Boston Tech Scene
Boston is a powerhouse for technology innovation thanks to world-class research universities like MIT and Harvard and a robust pipeline of venture capital investment. Host to the first telephone call and one of the first general-purpose computers ever put into use, Boston is now a hub for biotechnology, robotics and artificial intelligence — though it’s also home to several B2B software giants. So it’s no surprise that the city consistently ranks among the greatest startup ecosystems in the world.
Key Facts About Boston Tech
- Number of Tech Workers: 269,000; 9.4% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Thermo Fisher Scientific, Toast, Klaviyo, HubSpot, DraftKings
- Key Industries: Artificial intelligence, biotechnology, robotics, software, aerospace
- Funding Landscape: $15.7 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Summit Partners, Volition Capital, Bain Capital Ventures, MassVentures, Highland Capital Partners
- Research Centers and Universities: MIT, Harvard University, Boston College, Tufts University, Boston University, Northeastern University, Smithsonian Astrophysical Observatory, National Bureau of Economic Research, Broad Institute, Lowell Center for Space Science & Technology, National Emerging Infectious Diseases Laboratories
