Lead the development of data pipelines and transformations in Azure Databricks, converting Scala programs to PySpark while leveraging various Azure technologies.
Job Description :
Location - US (remote) . East coast preferable
Primary Skill :
Technical Lead -Data Engneering
- Azure databricks- Build Data Pipelines using ADB-Leveraging Python and Spark
- Build Data Transformation in ADB
- Convert Scala Pgms to Py Spark Pgms
- Understand the SCALA program in DataBricks
- ADF
- Secondary Skills:
- Azure data lake gen 2
- Synapse analytics (dedicated SQL pool)
- Exp in delta Lake
- Pyspark
Nice to Have : Tidal Scheduler , Exp in Healthcare Domain
Hexaware Technologies Boston, Massachusetts, USA Office
Boston, United States
Similar Jobs
Insurance
Leads engineering support and architecture for enterprise-scale healthcare data platforms on Azure. Designs Databricks, Synapse, Data Factory, and Data Lake solutions; builds batch and real-time pipelines, analytical data models, BI solutions, and automated data quality frameworks. Implements machine learning and AI-assisted development, integrates low-code systems, ensures HIPAA compliance, and translates business needs into technical solutions. Mentors engineers, resolves technical issues, supports Agile delivery, and partners with healthcare business leaders on Medicare and Medicaid analytics initiatives.
Top Skills:
Arm TemplatesAzure Data FactoryAzure Data Lake StorageAzure DatabricksAzure DevopsAzure Event HubsAzure Machine LearningAzure SynapseGithub CopilotHipaaKafkaAzureMicrosoft O365PegaPower BIPysparkPythonSalesforceScalaSitecoreSpark StreamingSQLTableauTerraform
Information Technology • Consulting
Lead development of enterprise data and analytics systems using Snowflake and Python. Build and optimize data pipelines on AWS (Lambda, Step Functions, ECS), write advanced SQL, and apply domain knowledge of asset management and financial markets.
Top Skills:
Amazon EcsAws LambdaAws Step FunctionsPythonSnowflakeSQL
Information Technology • Consulting
Lead data engineer responsible for designing, building, and operating production data pipelines using Python, Airflow, Snowflake, DBT, and AWS. Automate ETL, implement CI/CD, testing and DBT validations, troubleshoot pipelines, and work client-facing to deliver reliable data solutions.
Top Skills:
Ai IdeApache AirflowAws EcsAws IamAws S3Ci/CdCsvDbtGitJinjaJSONPythonSnowflakeSQLXML
What you need to know about the Boston Tech Scene
Boston is a powerhouse for technology innovation thanks to world-class research universities like MIT and Harvard and a robust pipeline of venture capital investment. Host to the first telephone call and one of the first general-purpose computers ever put into use, Boston is now a hub for biotechnology, robotics and artificial intelligence — though it’s also home to several B2B software giants. So it’s no surprise that the city consistently ranks among the greatest startup ecosystems in the world.
Key Facts About Boston Tech
- Number of Tech Workers: 269,000; 9.4% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Thermo Fisher Scientific, Toast, Klaviyo, HubSpot, DraftKings
- Key Industries: Artificial intelligence, biotechnology, robotics, software, aerospace
- Funding Landscape: $15.7 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Summit Partners, Volition Capital, Bain Capital Ventures, MassVentures, Highland Capital Partners
- Research Centers and Universities: MIT, Harvard University, Boston College, Tufts University, Boston University, Northeastern University, Smithsonian Astrophysical Observatory, National Bureau of Economic Research, Broad Institute, Lowell Center for Space Science & Technology, National Emerging Infectious Diseases Laboratories
