Design, build, and maintain AWS-based data pipelines and ETL processes using Glue, Spark/PySpark and EMR. Integrate multiple data sources into data lakes and Redshift, implement DevOps CI/CD practices, and work with Kafka, Hudi, and various AWS storage and analytics services.
This is a remote position.
- 3+ Years of Development / Data Engineering Experience
- 2+ Years Developing with AWS Cloud solutions
- 2+ Years Developing in Spark/PySpark
- 2+ Years Developing AWS Glue ETL
- 2 + Years AWS Storage models (S3 and DynamoDB)
Some hands-on developing with on-prem ETL tooling (Ab Initio preferred, Informatica)
Requirements
- Bachelor of degree in computer or any related field.
- Minimum of 12+ years’ experience in the said role.
- Strong experience in AWS GLUE, EMR, Hudi, experience extracting data from multiple sources & loading data into Data Lakes, AWS Redshift
- Experience working with AWS Elastic search, RDS, PostgreSQL preferred
- Experience w/ AWS services such as Lambda, EMR, SNS/SQS, Event Bridge, Lake Formation & Athena
- Experience w/ integrating applications/systems (data producers) with Enterprise Kafka topics (Confluent Kafka integration w/ AWS S3, Redshift)
- Experience in Requirements Analysis, Data Analysis, Application Design, Application Development & Integration Testing
- Working knowledge of on-prem Extraction, Transformation, Cleansing and Loading methodology and principles
- Experience implementing and adding to DevOps principles (GitLab, maintaining CI/CD pipelines)
- Experience in Java svcs is nice to have
N/A:- Please kindly note that this role is for USC and GC Visa holders only.
Interested and qualified candidates should send their resume to [email protected] using the job title as the subject of the mail.
Similar Jobs
Information Technology • Software
Own and scale AWS OpenSearch production infrastructure and PostgreSQL-to-OpenSearch data pipelines. Design indices and mappings, optimize queries and bulk indexing, monitor cluster health, implement ingestion reliability, perform reindexing/backfills, build reconciliation and recovery processes, and collaborate with backend, DB, and product teams on searchable schemas and runbooks.
Top Skills:
Amazon S3Amazon SqsAws DmsAws EcsAws EksAws LambdaAws Opensearch ServiceClickhouseCloudwatchElasticsearchGoIamJavaOpensearchOpensearch IngestionOpensearch Query DslPostgresPythonTerraformTinybird
Information Technology • Software
Own and scale AWS OpenSearch infrastructure and PostgreSQL-to-OpenSearch pipelines. Design indices, mappings, analyzers, and retention policies. Optimize queries and bulk indexing, monitor cluster health, implement retries/idempotency, backfills, reconciliation, and disaster recovery. Build dashboards, alerts, runbooks, and participate in incident response while collaborating with backend, DB, infra, and product teams.
Top Skills:
Aws DmsAws LambdaAws Opensearch ServiceClickhouseCloudwatchEcsEksElasticsearchGoIamJavaJvmOpensearchOpensearch IngestionOpensearch Query DslPostgresPythonS3SqsTerraformTinybird
Gaming • Software
Design, build, and maintain scalable AWS-based data pipelines and storage (S3, Redshift, RDS, DynamoDB). Implement IaC (CloudFormation/CDK), ETL optimization, monitoring, security (IAM, OWASP/CIS best practices), and CI/CD/code pipeline controls. Collaborate with product teams, document solutions, and provide training to support analytics and reliable data infrastructure.
Top Skills:
Amazon DynamodbAmazon KinesisAmazon RdsAmazon RedshiftAmazon S3AWSAws CdkAws CloudformationAws Data PipelineAws GlueAws IamCis Aws Foundations BenchmarkCweETLInfrastructure-As-CodeMonitoring And LoggingOwaspStatic Code Analysis
What you need to know about the Boston Tech Scene
Boston is a powerhouse for technology innovation thanks to world-class research universities like MIT and Harvard and a robust pipeline of venture capital investment. Host to the first telephone call and one of the first general-purpose computers ever put into use, Boston is now a hub for biotechnology, robotics and artificial intelligence — though it’s also home to several B2B software giants. So it’s no surprise that the city consistently ranks among the greatest startup ecosystems in the world.
Key Facts About Boston Tech
- Number of Tech Workers: 269,000; 9.4% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Thermo Fisher Scientific, Toast, Klaviyo, HubSpot, DraftKings
- Key Industries: Artificial intelligence, biotechnology, robotics, software, aerospace
- Funding Landscape: $15.7 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Summit Partners, Volition Capital, Bain Capital Ventures, MassVentures, Highland Capital Partners
- Research Centers and Universities: MIT, Harvard University, Boston College, Tufts University, Boston University, Northeastern University, Smithsonian Astrophysical Observatory, National Bureau of Economic Research, Broad Institute, Lowell Center for Space Science & Technology, National Emerging Infectious Diseases Laboratories
