The Staff Product Support Engineer will design, optimize, and manage Hadoop and Spark applications, solve data challenges, and ensure operational stability in a 24/7 environment.
We are seeking a Staff Product Support Engineer / Product Specialist - Hadoop Operations SME who will be responsible for designing, optimizing, migrating, and scaling Hadoop and Spark-based data processing systems. This role involves hands-on experience with Hadoop and other core data operations, focusing on building resilient, high-performance distributed data systems.
You will collaborate with customer engineering teams to deliver high-throughput Hadoop, Nifi, and Spark applications and solve complex data challenges in migration, upgrades, reliability, and optimise post-migration system performance.
This role requires flexibility to work in rotational shifts, based on team coverage needs and customer demand. Candidates should be comfortable supporting operations in a 24/7 environment and be willing to adjust their working hours accordingly.
We’re looking for someone who perform:
- Design and optimise distributed Hadoop-based applications, ensuring low-latency, high-throughput performance for big data workloads.
- Troubleshooting: Provide expert-level support for data or performance issues in Hadoop, Nifi, and Spark jobs and clusters.
- Data Processing Expertise: Work extensively with large-scale data pipelines using Hadoop, Nifi, and Spark's core components.
- Performance Tuning: Conduct deep-dive performance analysis, debugging, and optimization of Nifi, Impala, and Spark jobs to reduce processing time and resource consumption.
- Cluster Management: Collaborate with DevOps and infrastructure teams to manage Nifi, Impala, and Spark clusters on platforms like Hadoop/YARN, Kubernetes, or cloud platforms (AWS EMR, GCP Dataproc, etc.).
- Migration: Provide dedicated support to ensure the stability and reliability of the new ODP Hadoop environment during and after migration. Promptly address evolving technical challenges and optimise system performance after migration to ODP.
Good to have:
- Master’s degree and Experience working with scripting languages (Scala, Python, Bash, PowerShell).
- Familiarity with virtual machine technologies and multi-node environment (50+ nodes).
- Proficient with Linux, NFS, and Windows, including application installation, scripting, and working with the command line.
- Working knowledge of application, server, and network security management concepts. Certification on any of the leading Cloud providers (AWS, Azure, GCP ) and/or Kubernetes.
- Knowledge of databases like MySQL and PostgreSQL.
- Be involved with and work on other support-related activities - Performing POC & assisting with Onboarding deployments of Acceldata & Hadoop distribution products.
Top Skills
Aws Emr
Bash
Gcp Dataproc
Hadoop
Impala
Kubernetes
Linux
MySQL
Nifi
Postgres
Powershell
Python
Scala
Spark
Similar Jobs
Digital Media • Fintech • Information Technology • Machine Learning • Financial Services • Cybersecurity • Automation
The Private Client Financial Advisor develops personalized wealth management strategies for clients, enhances partnerships, and drives client engagement.
Top Skills:
Insurance HealthInsurance LifeInsurance VariableSeries 63Series 65Series 66Series 7
Aerospace • Cloud • Digital Media • Information Technology • Mobile • News + Entertainment • Generative AI
Handle 100% inbound customer contacts to sell EchoStar TV, internet, and wireless products using a consultative approach. Meet quotas in a fast-paced environment, maintain customer relationships with integrity, and work varied hours including evenings and weekends. Opportunity for rapid advancement and uncapped commission.
Top Skills:
2+ MonitorsHeadsetSmartphoneUsb KeyboardUsb MouseWebcamWired Internet (25 Mbps Download/2 Mbps Upload)
Aerospace • Information Technology • Software • Cybersecurity • Design • Defense • Manufacturing
Develop and maintain embedded software for avionics systems, ensuring compliance with standards, and integrating software with hardware components.
Top Skills:
BashCContinuous DeploymentContinuous IntegrationDistributed Version Control SystemsHardware DebuggersLinuxPythonReal-Time Operating Systems
What you need to know about the Boston Tech Scene
Boston is a powerhouse for technology innovation thanks to world-class research universities like MIT and Harvard and a robust pipeline of venture capital investment. Host to the first telephone call and one of the first general-purpose computers ever put into use, Boston is now a hub for biotechnology, robotics and artificial intelligence — though it’s also home to several B2B software giants. So it’s no surprise that the city consistently ranks among the greatest startup ecosystems in the world.
Key Facts About Boston Tech
- Number of Tech Workers: 269,000; 9.4% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Thermo Fisher Scientific, Toast, Klaviyo, HubSpot, DraftKings
- Key Industries: Artificial intelligence, biotechnology, robotics, software, aerospace
- Funding Landscape: $15.7 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Summit Partners, Volition Capital, Bain Capital Ventures, MassVentures, Highland Capital Partners
- Research Centers and Universities: MIT, Harvard University, Boston College, Tufts University, Boston University, Northeastern University, Smithsonian Astrophysical Observatory, National Bureau of Economic Research, Broad Institute, Lowell Center for Space Science & Technology, National Emerging Infectious Diseases Laboratories
