Leads the design, development, and delivery of scalable cloud data solutions using Snowflake, AWS, Python, PySpark, SQL, and big data technologies. Oversees project delivery, requirements gathering, architecture, development, testing, deployment, performance tuning, risk management, and stakeholder communication. Provides technical guidance to teams and supports cloud migrations, Agile delivery, and adoption of modern data engineering practices.
Lead Data Engineer with hands-on experience in designing and delivering scalable data solutions. The role requires expertise in cloud data platforms, AWS, Snowflake, data pipelines, ETL/ELT, data modernization, along with strong stakeholder management
Base Compensation Range: 110,000 - 140,000
The posted range is the hiring range for this role — a subset of the broader range available to employees over time — and reflects base salary across our national hiring scale. Final offers are based on several factors, including the candidate's skills and experience, internal pay equity, work location, market conditions for the role, and the specific scope and responsibilities of the position. The top of the range is reserved for candidates who notably exceed the requirements; the lower end applies to those with less experience or fewer preferred qualifications. For positions based in higher-cost zones (e.g., California, New York, New Jersey), actual compensation may exceed the posted range; your recruiter will share specifics during the process.
For more information on benefits and what we offer please visit us at https://www.exlservice.com/us-careers-and-benefits
Responsibilities- 7+ years of relevant experience in Data Engineering and project delivery.
- Strong expertise in Big Data concepts and cloud implementation projects.
- Proficient in Snowflake, SQL, and AWS services (Glue, EMR, S3, Aurora, RDS, AWS architecture).
- Excellent hands-on experience with Python, PySpark, and AWS cloud microservices.
- Capability to perform coding, debugging, performance tuning, and deploying applications to production environments.
- Strong problem-solving abilities, analytical skills, and aptitude for ownership.
- Skilled in Agile Methodology with experience in iterative project deliveries.
- Quick learner with the ability to lead team members in adopting new technologies.
- Exceptional communication and stakeholder management skills.
Good-to-Have Skills:
- Familiarity with DevOps tools and practices (e.g., Jenkins, Git) and experience in CI/CD pipelines.
- Knowledge of Spark, SQL (with Snowflake exposure), and advanced Big Data Concepts.
- Experience with cloud migrations and development projects
- Knowledge of US insurance and reinsurance domain
- Knowledge of Data Vault 2.0
Role & Responsibilities:
- Lead and oversee project delivery, ensuring timely completion with high quality.
- Collaborate with clients and offshore leads to gather requirements and define high-level designs.
- Direct development and unit testing activities while maintaining strong alignment with project goals.
- Communicate effectively with stakeholders, providing regular updates on task and project statuses.
- Identify, track, and address project risks or issues promptly.
- Work closely with management to ensure seamless execution of tasks.
- Provide technical guidance to the team, outlining design, implementation, and delivery strategies.
Qualifications
bachelors in engineering or related field
Similar Jobs
Computer Vision • Software
Leads the migration of legacy data warehouses and pipelines to an Azure and Databricks lakehouse. Defines technical architecture, reusable Azure Data Factory and Databricks pipelines, reconciliation methods, quality gates, and performance solutions. Mentors data engineers, reviews technical work, and collaborates with product, QA, platform, and customers in a remote Agile environment.
Top Skills:
AgileSparkAzureAzure Data FactoryDatabricksDelta LakePythonSQL
Aerospace • Security • Cybersecurity • Industrial
Leads the design, development, governance, and productization of enterprise data models, pipelines, data products, and ontologies. Builds machine-learning-ready datasets, implements security and lineage controls, optimizes Databricks and Delta Lake environments, establishes data-product standards, collaborates with architects and data scientists, and mentors engineering talent. The role also defines product roadmaps, service levels, usage metrics, and continuous-improvement processes for enterprise data assets supporting AI agents, analytics, and applications.
Top Skills:
Apache AirflowApache NifiSparkCloud IamDatabricksDelta LakeFeature StoresGenerative AiMachine Learning PipelinesPysparkPythonScalaSQLUnity Catalog
Information Technology
Lead enterprise data engineering efforts supporting defense missions. Responsibilities include prioritizing technical tasks, mentoring engineers, conducting code reviews, translating government requirements into scalable data architectures, developing Databricks ETL/ELT pipelines with Python, PySpark, and SQL, troubleshooting pipeline performance and reliability, and documenting systems. The role collaborates with government stakeholders, product owners, data scientists, analysts, and technical teams to deliver secure, high-throughput data platforms and actionable insights.
Top Skills:
SparkDatabricksGitlabJIRAPysparkPythonSQL
What you need to know about the Boston Tech Scene
Boston is a powerhouse for technology innovation thanks to world-class research universities like MIT and Harvard and a robust pipeline of venture capital investment. Host to the first telephone call and one of the first general-purpose computers ever put into use, Boston is now a hub for biotechnology, robotics and artificial intelligence — though it’s also home to several B2B software giants. So it’s no surprise that the city consistently ranks among the greatest startup ecosystems in the world.
Key Facts About Boston Tech
- Number of Tech Workers: 269,000; 9.4% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Thermo Fisher Scientific, Toast, Klaviyo, HubSpot, DraftKings
- Key Industries: Artificial intelligence, biotechnology, robotics, software, aerospace
- Funding Landscape: $15.7 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Summit Partners, Volition Capital, Bain Capital Ventures, MassVentures, Highland Capital Partners
- Research Centers and Universities: MIT, Harvard University, Boston College, Tufts University, Boston University, Northeastern University, Smithsonian Astrophysical Observatory, National Bureau of Economic Research, Broad Institute, Lowell Center for Space Science & Technology, National Emerging Infectious Diseases Laboratories
