TinyFish Logo

TinyFish

MLOps Engineer

Reposted 12 Days Ago
Remote or Hybrid
Hiring Remotely in United States
Senior level
Remote or Hybrid
Hiring Remotely in United States
Senior level
Build and maintain reproducible data pipelines, experiment orchestration, CI/CD for models, Terraform-based ML infrastructure, observability, security controls, and automation to deploy and operate ML systems in production.
The summary above was generated by AI
Position Overview

As the first dedicated ML Ops Engineer, you’ll own the tooling and infrastructure that make our ml engineers wildly productive and ensure we are able to efficiently iterate on ML models, prompts, and datasets and deploy our AI systems into a predictable production environment. You’ll bridge the gap between research and DevOps—designing reproducible dataset pipelines, automated experiment workflows, and Terraform-based cloud deployments that scale.

Key Responsibilities

Dataset Management

• Design version-controlled data pipelines (feature stores, data registries) using tools such as Delta Lake, Apache Iceberg
• Implement systems for data validation, lineage tracking, and automated quality checks (e.g., Great Expectations).

Experiment Execution & Tracking

• Build and maintain experiment orchestration with platforms like MLflow, torchx, and Apache Airflow.
• Provide templated systems and tools to ML Engineers that easily launch training/evaluation data processing systems
• Automate hyper-parameter sweeps and A/B tests, exposing clear dashboards for results.

CI/CD

Models/Agents

• workflows that package, test, and promote models and agents through staging to production.
• Implement canary deployments and rollbacks for models/agents services

Terraform Infrastructure-as-Code•

• Author and maintain Terraform modules for all ML infra—networking, GPU/TPU clusters, object storage, secrets, monitoring.
• Enforce best practices for state management, workspaces, and automated plan/apply stages via CI.

Observability & Reliability

• Integrate logging, tracing, and metric collection (Prometheus, Grafana, Datadog) across data pipelines and model endpoints.
• Set SLIs/SLOs for data freshness and model latency; implement alerts and runbooks.

Security & Compliance• Work with Security to implement IAM least-privilege, key rotation, and data-encryption policies.
• Support audit requirements (SOC 2, GDPR, HIPAA where applicable).

Minimum Qualifications
  • 5+ years combined experience in DevOps, Data Engineering, or ML Ops roles.

  • Strong Terraform skills; ability to craft reusable modules and navigate complex state.

  • Production experience with at least one cloud provider (AWS, GCP, or Azure).

  • Proficiency in Python and containerization (Docker); familiarity with Kubernetes or serverless batch systems.

  • Hands-on knowledge of ML experiment platforms (MLflow, Kubeflow, Weights & Biases, or similar).

  • Experience with workflow execution frameworks (Kubeflow, Apache Airflow)

  • Understanding of modern data-versioning/feature-store concepts and tools.

  • Solid grasp of CI/CD principles, Git workflows, and infrastructure testing.

  • Excellent communication skills—capable of partnering with Data Scientists, Software Engineers, and Security teams.

Preferred (Nice-to-Have)
  • Experience with GPU orchestration (NVIDIA DGX, Karpenter, or Ray).

  • Familiarity with IaC security scanning (Checkov, tfsec).

  • Exposure to policy-as-code (OPA/Gatekeeper).

  • Prior work in real-time streaming (Kafka, Flink) and online feature serving.

  • Contributions to open-source ML Ops projects.

Reporting Structure

Reports to: Director of Infra

Similar Jobs

A Minute Ago
In-Office or Remote
Junior
Junior
Healthtech • Insurance
Build, deploy, monitor, automate, and optimize machine learning solutions in production. Collaborate with data scientists, software engineers, and IT operations on model integration, version control, governance, data pipelines, and cloud-based ML workflows. The role also involves supporting full-stack applications, troubleshooting infrastructure and models, and applying MLOps best practices across the machine learning lifecycle.
Top Skills: Automated TestingAWSAzureCi/CdDatabricksGCPGitKubernetesMlflowOpenshiftPythonSQLTerraform
14 Days Ago
Remote
United States
90-120 Hourly
Junior
90-120 Hourly
Junior
Artificial Intelligence • HR Tech • Professional Services • Software
Build and evaluate MLOps and ML systems tasks for frontier AI training data. Responsibilities include designing technical challenges, writing solutions and evaluation rubrics, profiling and optimizing GPU workloads, debugging distributed systems, improving model performance, and supporting high-throughput LLM inference. The role requires production experience with ML infrastructure, serving systems, GPU accelerators, JAX or PyTorch, and strong technical communication.
Top Skills: A100B200Continuous BatchingCudaDdpDeepspeedFsdpH100JaxKinetoKv CacheMegatronNsightPaged AttentionPallasPyTorchRay ServeSglangTensorrt-LlmTorch.ProfilerTpuTritonVllmXla
17 Days Ago
In-Office or Remote
California, USA
152K-230K Annually
Senior level
152K-230K Annually
Senior level
Software • Travel
Build and maintain internal platforms and tooling for AI engineers, including React and Next.js dashboards, Python backend services, REST APIs, model deployment automation, observability integrations, and ML pipeline testing. Partner with engineering and ML teams to improve developer experience, production AI reliability, and self-service workflows while documenting systems and runbooks.
Top Skills: AWSAzureDatabasesDatadogDockerGCPGrafanaInfrastructure As Code (Iac)Job Queuing SystemsKubernetesLangsmithNext.JsPrometheusPythonReactRest ApisTypescript

What you need to know about the Boston Tech Scene

Boston is a powerhouse for technology innovation thanks to world-class research universities like MIT and Harvard and a robust pipeline of venture capital investment. Host to the first telephone call and one of the first general-purpose computers ever put into use, Boston is now a hub for biotechnology, robotics and artificial intelligence — though it’s also home to several B2B software giants. So it’s no surprise that the city consistently ranks among the greatest startup ecosystems in the world.

Key Facts About Boston Tech

  • Number of Tech Workers: 269,000; 9.4% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Thermo Fisher Scientific, Toast, Klaviyo, HubSpot, DraftKings
  • Key Industries: Artificial intelligence, biotechnology, robotics, software, aerospace
  • Funding Landscape: $15.7 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Summit Partners, Volition Capital, Bain Capital Ventures, MassVentures, Highland Capital Partners
  • Research Centers and Universities: MIT, Harvard University, Boston College, Tufts University, Boston University, Northeastern University, Smithsonian Astrophysical Observatory, National Bureau of Economic Research, Broad Institute, Lowell Center for Space Science & Technology, National Emerging Infectious Diseases Laboratories

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account