Designs and leads production-grade batch, incremental, streaming, and real-time data pipelines across Microsoft Fabric and Azure. Builds Delta Lake medallion architectures, dimensional and Data Vault models, CDC and incremental-loading solutions, CI/CD automation, governance, security, observability, and data-quality controls. Supports migrations to Fabric, prepares governed data for AI and RAG workloads, and leads client-facing engineering workstreams, estimates, risk management, mentoring, and operational handover.
- 8+ years in data engineering/data warehousing; 3+ years delivering Azure solutions and production
- Microsoft Fabric experience
- Advanced T-SQL, Spark SQL, Python/PySpark; working knowledge of KQL and DAX
- Deep experience with Delta Lake, Spark tuning, medallion/lakehouse patterns, CDC and incremental loading
- Strong dimensional modeling and working knowledge of Data Vault 2.0
- Practical CI/CD, Git, automated testing, infrastructure as code, DataOps, governance and security experience
- Professional-services/client-facing delivery experience
- Authorization to work in the US and ability to travel up to 15%
- Preferred: DP-700, DP-600, DP-203; streaming/event-driven, RAG/GenAI, Fabric IQ/data agents, and industry experience
- Design, build and maintain production-grade batch, incremental, real-time and near-real-time data pipelines across Microsoft Fabric and Azure
- Implement CDC, watermarking, merge/upsert, late-arriving-data and idempotent reprocessing patterns
- Develop PySpark, Spark SQL and T-SQL transformations across bronze, silver and gold Delta Lake layers
- Build Fabric solutions using OneLake, Lakehouse, Warehouse, Data Factory, Dataflows Gen2, Spark notebooks, Real-Time Intelligence and Power BI
- Implement dimensional and Data Vault models, data-quality testing, reconciliation, observability, logging, alerting and SLA monitoring
- Establish Git-based CI/CD and automated deployment using Azure DevOps or GitHub, Fabric deployment pipelines, REST APIs, Bicep and Terraform
- Apply Purview catalog/lineage, least-privilege access, sensitivity controls, encryption and secure connectivity
- Support migrations from SQL Server, SSIS, SSAS, legacy warehouses and Synapse to Fabric/Azure, including conversion, reconciliation and cutover
- Prepare governed data for AI/ML and RAG workloads, including ingestion, chunking, embeddings, indexing and telemetry
- Lead engineering workstreams, estimates, risk management, client communication, mentoring, reusable accelerators and operational handover
Similar Jobs
Blockchain • Fintech • Payments • Financial Services • Cryptocurrency • Web3
Build and operate production data pipelines, models, and data products supporting finance, accounting, treasury, compliance, and regulatory reporting. Partner cross-functionally to define requirements, model on-chain and internal data, and ensure data reliability through reconciliation, quality checks, monitoring, and alerting. Use SQL, Python, cloud data warehouse, Airflow, dbt, and AI tooling. Mentor engineers, review technical work, document systems, and communicate data concepts to technical and non-technical audiences.
Top Skills:
Apache AirflowBlockchainCloud Data WarehouseDbtPythonSQL
Fintech • Software • Financial Services
Own data pipelines into and out of the warehouse, dbt models, data integrity, governance, identity resolution, reverse ETL, and quality monitoring across growth, lifecycle, and brokerage data. Partner with product, marketing, and operations teams to establish data contracts, ensure accurate and timely business metrics, and support regulated financial data requirements.
Top Skills:
AirflowArgo WorkflowsChange Data CaptureDatadogDbtFivetranKubernetesPythonReverse EtlSegmentSnowflakeSQL
Cloud • Information Technology • Security • Software • Cybersecurity
Deploys, maintains, troubleshoots, and scales physical server, network, and cabling infrastructure across production data centers. Responsibilities include rack-and-stack deployments, structured cabling, hardware repairs, network fault remediation, ticket and corrective maintenance management, asset documentation, and coordination with engineering teams and remote-hands vendors. The role requires hands-on technical work, lifting up to 50 pounds, U.S. citizenship, and up to 50% domestic and international travel.
Top Skills:
Ai-Assisted Engineering ToolsAutomationBiosCopper CablingFiber OpticsIdracIloIpmiLinux/UnixNetwork HardwareScriptingServer PlatformsStructured Cabling
What you need to know about the Boston Tech Scene
Boston is a powerhouse for technology innovation thanks to world-class research universities like MIT and Harvard and a robust pipeline of venture capital investment. Host to the first telephone call and one of the first general-purpose computers ever put into use, Boston is now a hub for biotechnology, robotics and artificial intelligence — though it’s also home to several B2B software giants. So it’s no surprise that the city consistently ranks among the greatest startup ecosystems in the world.
Key Facts About Boston Tech
- Number of Tech Workers: 269,000; 9.4% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Thermo Fisher Scientific, Toast, Klaviyo, HubSpot, DraftKings
- Key Industries: Artificial intelligence, biotechnology, robotics, software, aerospace
- Funding Landscape: $15.7 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Summit Partners, Volition Capital, Bain Capital Ventures, MassVentures, Highland Capital Partners
- Research Centers and Universities: MIT, Harvard University, Boston College, Tufts University, Boston University, Northeastern University, Smithsonian Astrophysical Observatory, National Bureau of Economic Research, Broad Institute, Lowell Center for Space Science & Technology, National Emerging Infectious Diseases Laboratories
