Enhance a Python-based ingestion framework for operational metadata and build connectors for major systems in data and ML stacks.
DataHub is an AI & Data Context Platform adopted by over 3,000 enterprises, including Apple, CVS Health, Netflix, and Visa. Innovated jointly with a thriving open-source community of 13,000+ members, DataHub's metadata graph provides in-depth context of AI and data assets with best-in-class scalability and extensibility.
The company's enterprise SaaS offering, DataHub Cloud, delivers a fully managed solution with AI-powered discovery, observability, and governance capabilities. Organizations rely on DataHub solutions to accelerate time-to-value from their data investments, ensure AI system reliability, and implement unified governance, enabling AI & data to work together and bring order to data chaos.
In this role, you will
- Enhance the Python-based ingestion framework to support ingesting usage statistics, lineage, and operational metadata from systems like Snowflake, Redshift, Kafka, & more!
- Build connectors for major systems in the modern data and ML stacks
- Enable the ingestion framework to run in a cloud native environment
Requirements:
- Minimum 4 years of engineering experience
- Expertise in Python
- Familiarity with tools in the modern data and ML ecosystem
- Knowledge of distributed systems
- Ability to design for scale and fault tolerance
Benefits
- Competitive salary
- Equity
- Medical, dental, and vision insurance (99% coverage for employees, 65% coverage for dependents; USA-based employees)
- Carrot Fertility Program (USA-based employees)
- Work from home and monthly co-working space budget
Top Skills
Kafka
Python
Redshift
Snowflake
Similar Jobs
Fintech • Machine Learning • Payments • Software • Financial Services
As a Senior Software Engineer, you'll lead technology projects, develop cloud-native solutions, write code, promote engineering excellence, and mentor others.
Top Skills:
AWSDockerGCPGoJavaJavaScriptKubernetesAzureNode.jsPythonRustTypescript
Artificial Intelligence • eCommerce • Food
The Senior Frontend Software Engineer will design, implement, and maintain the consumer-facing web interface, collaborate with stakeholders, test code, and enhance code quality.
Top Skills:
Ci/CdCSSDatadogDockerGithub ActionsJIRANginxPlaywrightReactSentryTypescriptVitest
Blockchain • eCommerce • Fintech • Payments • Software • Financial Services • Cryptocurrency
As a Software Engineer, you'll build and scale systems for Cash App, participate in development and technical discussions, and support millions of customers.
Top Skills:
AWSDynamoDBGrpcHTTPJavaJSONKafkaKotlinKubernetesProtocol BuffersVitess
What you need to know about the Boston Tech Scene
Boston is a powerhouse for technology innovation thanks to world-class research universities like MIT and Harvard and a robust pipeline of venture capital investment. Host to the first telephone call and one of the first general-purpose computers ever put into use, Boston is now a hub for biotechnology, robotics and artificial intelligence — though it’s also home to several B2B software giants. So it’s no surprise that the city consistently ranks among the greatest startup ecosystems in the world.
Key Facts About Boston Tech
- Number of Tech Workers: 269,000; 9.4% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Thermo Fisher Scientific, Toast, Klaviyo, HubSpot, DraftKings
- Key Industries: Artificial intelligence, biotechnology, robotics, software, aerospace
- Funding Landscape: $15.7 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Summit Partners, Volition Capital, Bain Capital Ventures, MassVentures, Highland Capital Partners
- Research Centers and Universities: MIT, Harvard University, Boston College, Tufts University, Boston University, Northeastern University, Smithsonian Astrophysical Observatory, National Bureau of Economic Research, Broad Institute, Lowell Center for Space Science & Technology, National Emerging Infectious Diseases Laboratories
