The AI Data Engineer will develop and manage ETL pipelines, ensure quality in data workflows, and support AI/ML integration within autonomous systems.
Architect, build, and operate real-time/batch ETL pipelines, agentic orchestration flows, and AI/ML endpoints for autonomous, multi-agent production systems. Contribute actively to team processes, documentation, and operational quality.
Responsibilities
- Build event-driven data workflows (Snowflake, S3, Kafka, EventBridge, Celery, AWS Batch), integrate with FactSet, Sharepoint, Proxy connectors, and expose agentic features (LangChain, LangGraph, LlamaIndex, Pinecone).
- Develop, maintain, and monitor vector database (Pinecone) pipelines, LLM and ML endpoints, ensuring agent memory/state is managed for retrieval-augmented pipelines.
- Automate schema/version change management, event contract validation, lineage tracking, and observability.
- Write, run, and document QA/test coverage for ETL, agentic triggers, and GenAI model events, with incident and postmortem participation.
- Collaborate in agile ceremonies: propose improvements, troubleshoot delivery bottlenecks, and share technical knowledge via docs and training sessions.
- Implement and monitor security, compliance, and resiliency standards at all stages of the data/model workflows.
- Help onboard new team members; mentor engineers in agentic and event-driven data engineering best practices.
Qualifications
- Event-driven ETL/data pipeline experience (Kafka, EventBridge, Snowflake, S3, Python/Celery), plus direct work with LangChain/LangGraph/LlamaIndex/Pinecone multi-agent orchestration.
- QA, documentation, monitoring, and collaborative agile process skills.
- Experience with cross-cloud, multi-source integration and incident resolution.
Top Skills
Aws Batch
Celery
Eventbridge
Kafka
Langchain
Langgraph
Llamaindex
Pinecone
Python
S3
Snowflake
Similar Jobs
Artificial Intelligence • Information Technology • Marketing Tech • Software • SEO
In this full stack role, you will build features around data pipelines and LLMs, create web APIs, and work on analytics tools ensuring data quality and processing capabilities.
Top Skills:
BeamCeleryDuckdbFlinkGcp DataflowGoogle BigqueryGoogle Cloud TasksInngestLlmsPandasPolarsPostgresReactResqueSnowflakeSparkSpark SqlTemporal
.png)
Artificial Intelligence • Other • Security • Software • Analytics • Big Data Analytics
Lead Data & AI Engineer responsible for architecting and implementing intelligent analytics for a surveillance cloud platform, requiring strong skills in data pipelines, AI model management, and modern cloud technologies.
Top Skills:
DockerGCPGoGrpcIcebergK8SPythonSparkTerraformTriton
Greentech • Software
As a Data/AI Engineer, you'll build scalable data systems, automate data collection with AI tools, and analyze sustainability metrics to inform consumer choices.
Top Skills:
AILlmsPythonSQL
What you need to know about the Boston Tech Scene
Boston is a powerhouse for technology innovation thanks to world-class research universities like MIT and Harvard and a robust pipeline of venture capital investment. Host to the first telephone call and one of the first general-purpose computers ever put into use, Boston is now a hub for biotechnology, robotics and artificial intelligence — though it’s also home to several B2B software giants. So it’s no surprise that the city consistently ranks among the greatest startup ecosystems in the world.
Key Facts About Boston Tech
- Number of Tech Workers: 269,000; 9.4% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Thermo Fisher Scientific, Toast, Klaviyo, HubSpot, DraftKings
- Key Industries: Artificial intelligence, biotechnology, robotics, software, aerospace
- Funding Landscape: $15.7 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Summit Partners, Volition Capital, Bain Capital Ventures, MassVentures, Highland Capital Partners
- Research Centers and Universities: MIT, Harvard University, Boston College, Tufts University, Boston University, Northeastern University, Smithsonian Astrophysical Observatory, National Bureau of Economic Research, Broad Institute, Lowell Center for Space Science & Technology, National Emerging Infectious Diseases Laboratories
