Lead the development of data pipelines and transformations in Azure Databricks, converting Scala programs to PySpark while leveraging various Azure technologies.
Job Description :
Location - US (remote) . East coast preferable
Primary Skill :
Technical Lead -Data Engneering
- Azure databricks- Build Data Pipelines using ADB-Leveraging Python and Spark
- Build Data Transformation in ADB
- Convert Scala Pgms to Py Spark Pgms
- Understand the SCALA program in DataBricks
- ADF
- Secondary Skills:
- Azure data lake gen 2
- Synapse analytics (dedicated SQL pool)
- Exp in delta Lake
- Pyspark
Nice to Have : Tidal Scheduler , Exp in Healthcare Domain
Hexaware Technologies Boston, Massachusetts, USA Office
Boston, United States
Similar Jobs
Information Technology • Consulting
Leads the design and development of enterprise-wide data and analytics systems using Snowflake, Python, advanced SQL, and AWS services. Requires strong data engineering expertise and experience supporting financial services organizations, with knowledge of asset management and financial markets.
Top Skills:
Amazon EcsAWSAws LambdaAws Step FunctionsPythonSnowflakeSQL
Information Technology • Consulting
Leads the design and development of enterprise-wide data and analytics systems using Snowflake, Python, advanced SQL, and AWS services including Lambda, Step Functions, and ECS. Requires strong data engineering expertise and practical experience building modern platforms within financial services, with knowledge of asset management and financial markets.
Top Skills:
Amazon EcsAws LambdaAws Step FunctionsPythonSnowflakeSQL
Information Technology • Consulting
Leads hands-on big data processing, data engineering, and analytics workflows using Azure Databricks, Python, and SQL. Develops insights and reporting with Power BI, Tableau, and Excel. Analyzes complex datasets, communicates findings to business stakeholders, manages competing priorities, meets deadlines, and promotes continuous improvement.
Top Skills:
Azure DatabricksExcelPower BIPythonSQLTableau
What you need to know about the Boston Tech Scene
Boston is a powerhouse for technology innovation thanks to world-class research universities like MIT and Harvard and a robust pipeline of venture capital investment. Host to the first telephone call and one of the first general-purpose computers ever put into use, Boston is now a hub for biotechnology, robotics and artificial intelligence — though it’s also home to several B2B software giants. So it’s no surprise that the city consistently ranks among the greatest startup ecosystems in the world.
Key Facts About Boston Tech
- Number of Tech Workers: 269,000; 9.4% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Thermo Fisher Scientific, Toast, Klaviyo, HubSpot, DraftKings
- Key Industries: Artificial intelligence, biotechnology, robotics, software, aerospace
- Funding Landscape: $15.7 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Summit Partners, Volition Capital, Bain Capital Ventures, MassVentures, Highland Capital Partners
- Research Centers and Universities: MIT, Harvard University, Boston College, Tufts University, Boston University, Northeastern University, Smithsonian Astrophysical Observatory, National Bureau of Economic Research, Broad Institute, Lowell Center for Space Science & Technology, National Emerging Infectious Diseases Laboratories
