American Family Insurance Logo

American Family Insurance

Lead Data Engineer

Reposted Yesterday
Be an Early Applicant
In-Office
Boston, MA, USA
131K-220K Annually
Entry level
In-Office
Boston, MA, USA
131K-220K Annually
Entry level
Leads a data engineering team that builds scalable big data applications and ETL workflows. Oversees data quality, security, privacy, architecture, documentation, and operational efficiency. Develops reusable Python and SQL code for data warehouses, distributed processing, cloud platforms, and data-driven solutions. Collaborates with application architects, development teams, project teams, and business stakeholders while supervising team activities and meeting project priorities.
The summary above was generated by AI
This position collects, stores, processes and builds applications within our big data platform. Responsible for integrating these applications with the architecture used across the organization.

Position Compensation Range:

$131,000.00 - $220,000.00

Pay Rate Type:

Salary

Compensation may vary based on the job level and your geographic work location. Relocation support is offered for eligible candidates.

This position has approval to transfer a valid H1 visa.

Primary Accountabilities

  • Leads an engineering team to meet project deadlines and priorities.

  • Supervises assigned data engineering team members & activities.

  • Ensures the quality, completeness, security, privacy, and integrity of data throughout the data lifecycle.

  • Documents critical workflows and operational support aspects of team’s responsibilities

  • Develops deep understanding of data sources, granularity, availability, and limitations.

  • Provides proactive technical oversight and advice to application architecture and development teams fostering re-use, design for scale, stability, and operational efficiency of data/analytical solutions.

  • Creates maintainable, scalable code to load and manipulate data in the data warehouse.

  • Facilitates communication upward and across project teams and business stakeholders.

Specialized Knowledge & Skills Requirements

  • Demonstrated experience providing customer-driven solutions, support or service.

  • Must have GCP experience for this positon.

  • In-depth knowledge of SQL or NoSQL and experience using a variety of data stores (e.g. RDBMS, analytic database, scalable document stores)

  • Extensive hands-on Python programming experience, with an emphasis towards building ETL workflows and data-driven solutions. 

  • Able to employ design patterns and generalize code to address common use cases.  

  • Capable of authoring robust, high quality, reusable code and contributing to the division’s inventory of libraries.

  • Expertise in big data batch computing tools (e.g. Hadoop or Spark), with demonstrated experience developing distributed data processing solutions.

  • Knowledge of open source machine learning toolkits, such as sklearn, SparkML, or H2O.

  • Solid data understanding and business acumen in the data rich industries like insurance or financial

  • Applied knowledge of data modeling principles (e.g. dimensional modeling and star schemas).

  • Strong understanding of database internals, such as indexes, binary logging, and transactions.

  • Experience using tools for infrastructure-as-code (e.g. Docker, CloudFormation, Terraform, etc.)

  • Experience with software engineering tools and workflows (i.e. Jenkins, CI/CD, git).

  • Practical experience authoring and consuming web services.

Licenses:
•Not Applicable.
Travel Requirements
•Up to 10%.
Physical Requirements
•Work that primarily involves sitting/standing.
Working Conditions
•Not Applicable.

Additional Information
  • To ensure a strong start, all employees participate in our New Employee Orientation during their first week.  This experience is held in person at our Madison, WI Headquarters or one of our AmFam core locations to help you connect with our mission, meet key team members and build relationships that support your growth.  At times, sessions may be delivered virtually based on scheduling and availability. 
  • Offer to selected candidate will be made contingent on the results of applicable background checks
  • Offer to selected candidate is contingent on signing a non-disclosure agreement for proprietary information, trade secrets, and inventions
  • Sponsorship will not be considered for this position unless specified in the posting

We provide benefits that support your physical, emotional, and financial wellbeing. You will have access to comprehensive medical, dental, vision and wellbeing benefits that enable you to take care of your health. We also offer a competitive 401(k) contribution, a pension plan, an annual incentive, 9 paid holidays and a paid time off program (23 days accrued annually for full-time employees). In addition, our student loan repayment program and paid-family leave are available to support our employees and their families. Interns and contingent workers are not eligible for American Family Insurance Group benefits.

We are an equal opportunity employer. It is our policy to comply with all applicable federal, state and local laws pertaining to non-discrimination, non-harassment and equal opportunity. We also consider qualified applicants with criminal histories, consistent with applicable federal, state and local law.

American Family Insurance is committed to the full inclusion of all qualified individuals. If a reasonable accommodation is needed to participate in the job application or interview process, to perform essential job functions, and/or to receive other benefits and privileges of employment, please email [email protected] to request a reasonable accommodation.

#LI-BC2

Similar Jobs

One Month Ago
Remote or Hybrid
United States
Mid level
Mid level
Information Technology • Database • Consulting
Lead data engineering teams to design, build, and maintain scalable cloud-native data pipelines, data platforms, and data products using Snowflake, Databricks, Airflow, and modern architectures. Optimize ETL/ELT, SQL, and transformations for analytics, reporting, and AI use cases; ensure data quality, governance, observability, and stakeholder alignment. Produce documentation, runbooks, and present technical solutions to business and technical audiences.
Top Skills: Apache AirflowAWSAzureDatabricksGCPPysparkPythonSnowflakeSQL
23 Days Ago
Easy Apply
Remote or Hybrid
United States
Easy Apply
170K-205K Annually
Senior level
170K-205K Annually
Senior level
Fintech • Financial Services
Lead the Data Platform Engineering team owning ETL/reverse-ETL pipelines, replication, and production datastores. Define architecture, roadmap, standards (IaC, observability, schema contracts), mentor engineers, drive incident response and durable fixes, and partner on cross-team technical decisions and operational reliability for data correctness, completeness, and timeliness.
Top Skills: Agentic Ai Coding ToolsAirbyteAmazon AuroraAWSAzureEltETLFivetranGCPInformaticaInfrastructure As CodeObservabilityPostgresReverse-EtlRuby On RailsSQL
One Month Ago
Remote or Hybrid
United States
94K-154K Annually
Senior level
94K-154K Annually
Senior level
Information Technology • Database • Consulting
Lead Data Engineer builds and maintains scalable data pipelines and lakehouse/warehouse platforms (Databricks) to support analytics, BI, reporting, and AI. Responsibilities include data modeling, ETL/ELT, medallion architecture, orchestration and monitoring (Airflow/Cron), performance tuning, CDC/incremental loads, data quality, and leading/mentoring a team while collaborating with stakeholders and maintaining documentation.
Top Skills: Apache AirflowAWSAzureCronData WarehousingDatabricksEltETLGCPHadoopHbaseHiveLakehouseMedallion ArchitecturePigPower BIPysparkPythonSparkSQLTableau

What you need to know about the Boston Tech Scene

Boston is a powerhouse for technology innovation thanks to world-class research universities like MIT and Harvard and a robust pipeline of venture capital investment. Host to the first telephone call and one of the first general-purpose computers ever put into use, Boston is now a hub for biotechnology, robotics and artificial intelligence — though it’s also home to several B2B software giants. So it’s no surprise that the city consistently ranks among the greatest startup ecosystems in the world.

Key Facts About Boston Tech

  • Number of Tech Workers: 269,000; 9.4% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Thermo Fisher Scientific, Toast, Klaviyo, HubSpot, DraftKings
  • Key Industries: Artificial intelligence, biotechnology, robotics, software, aerospace
  • Funding Landscape: $15.7 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Summit Partners, Volition Capital, Bain Capital Ventures, MassVentures, Highland Capital Partners
  • Research Centers and Universities: MIT, Harvard University, Boston College, Tufts University, Boston University, Northeastern University, Smithsonian Astrophysical Observatory, National Bureau of Economic Research, Broad Institute, Lowell Center for Space Science & Technology, National Emerging Infectious Diseases Laboratories

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account