Similar Jobs
Artificial Intelligence • Insurance • Machine Learning • Software • Analytics
Build and maintain automated data quality testing for customer data pipelines. Responsibilities include developing dbt, Soda, and SQL checks; validating schemas, loads, sequencing, and downstream reconciliations; detecting anomalies; managing severity levels and customer overrides; reporting quality metrics; and partnering with engineering teams to embed testing into new pipelines. The role requires strong backend testing, SQL, Python, Airflow, and Databricks experience, along with excellent debugging and communication skills.
Top Skills:
Apache AirflowCi/CdCypressDatabricksDbtDelta LakeGitPlaywrightPythonSodaSpark SqlSQL
Cloud • Information Technology • Productivity • Security • Software • App development • Automation
Build and maintain scalable data pipelines, analytical data models, backend data systems, business rules, metrics, and data sources. Partner with teams across finance, growth, product, customer support, sales, and marketing to support analytical needs and improve data-driven decision-making. Own data engineering problems end to end and help advance organizational data practices.
Fintech • Machine Learning • Payments • Software • Financial Services
Design, build, and operate scalable cloud data pipelines, platforms, and applications using distributed data technologies. Lead architectural decisions, evaluate platforms such as Snowflake and Databricks, establish engineering patterns, and deliver resilient data solutions. Collaborate with product managers, software engineers, analysts, and data scientists while mentoring engineers and communicating technical outcomes to stakeholders. Drive large-scale data initiatives independently in an Agile environment.
Top Skills:
AgileAirflowAWSCassandraDagsterDatabricksDynamoDBEmrGlueGCPJavaAzureMongoDBMonte CarloNoSQLPythonRedshiftScalaSnowflakeSparkSplunkSQL
Build and maintain Python-based data ingestion and ETL pipelines for a large-scale migration. Extract data from legacy systems, APIs, databases, and enterprise platforms; transform, validate, and load it into MySQL and PostgreSQL. Monitor and optimize pipeline reliability, troubleshoot data issues, document technical solutions, and collaborate with engineering and business stakeholders. The role requires strong SQL, API, Git, data quality, securities data, and vulnerability data management experience.
Position Overview
We are seeking a highly experienced Senior Data Engineer with strong Python development skills to support a large-scale data migration initiative. This individual will build and maintain data ingestion pipelines that extract data from legacy and enterprise systems, transform it, and load it into SQL databases for downstream reporting and analytics.
The ideal candidate is a self-sufficient engineer who is comfortable working across APIs, databases, and ETL pipelines, with the ability to independently deliver high-quality solutions in a fast-paced environment.
- Design, develop, and maintain Python-based data ingestion and ETL pipelines.
- Extract data from legacy/homegrown securities platforms, REST APIs, MySQL, PostgreSQL, Microsoft System Center, MDM tools, and similar enterprise systems.
- Develop Python scripts to transform, validate, and load data into MySQL and PostgreSQL databases.
- Support migration of existing data feeds into a new environment while ensuring data quality and reliability.
- Monitor, troubleshoot, and optimize data pipelines.
- Collaborate with engineering and business stakeholders to implement scalable data solutions.
- Document data flows, processes, and technical solutions.
- 7+ years of Data Engineering and Python development experience.
- Expert-level Python programming.
Experience working with Fabric. - Strong ETL/data ingestion pipeline experience.
- Experience with APIs, MySQL, and PostgreSQL.
- Strong SQL skills.
- Experience using Git/GitHub.
- Securities and vulnerability type of data analysis/management experience
- Ability to work independently with minimal oversight.
- Strong analytical and troubleshooting skills.
Cleartelligence, Inc. Newton, Massachusetts, USA Office
275 Grove Street, Newton, MA, United States, 02466
What you need to know about the Boston Tech Scene
Boston is a powerhouse for technology innovation thanks to world-class research universities like MIT and Harvard and a robust pipeline of venture capital investment. Host to the first telephone call and one of the first general-purpose computers ever put into use, Boston is now a hub for biotechnology, robotics and artificial intelligence — though it’s also home to several B2B software giants. So it’s no surprise that the city consistently ranks among the greatest startup ecosystems in the world.
Key Facts About Boston Tech
- Number of Tech Workers: 269,000; 9.4% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Thermo Fisher Scientific, Toast, Klaviyo, HubSpot, DraftKings
- Key Industries: Artificial intelligence, biotechnology, robotics, software, aerospace
- Funding Landscape: $15.7 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Summit Partners, Volition Capital, Bain Capital Ventures, MassVentures, Highland Capital Partners
- Research Centers and Universities: MIT, Harvard University, Boston College, Tufts University, Boston University, Northeastern University, Smithsonian Astrophysical Observatory, National Bureau of Economic Research, Broad Institute, Lowell Center for Space Science & Technology, National Emerging Infectious Diseases Laboratories
