Machine Learning Engineer: Post Training and Speculative Decoding

Machine Learning Engineer: Training and Speculative Decoding

Mission: We are seeking a highly skilled Machine Learning Engineer to join our advanced model development team. This role focuses on pre-training, continued training, and post-training of models, with a particular emphasis on draft model optimization for speculative decoding and quantization-aware training (QAT). The ideal candidate has deep experience with training methodologies, open-weight models, and performance-tuning for inference.

Responsibilities & outcomes:

• Lead pre-training and post-training efforts for draft models tailored to speculative decoding architectures.
• Conduct continued training and post-training of open-weight models for non-draft (standard) inference scenarios.
• Implement and optimize quantization-aware training pipelines to enable low-precision inference with minimal accuracy loss.
• Collaborate with model architecture, inference, and systems teams to evaluate model readiness across training and deployment stages.
• Develop tooling and evaluation metrics for training effectiveness, draft model fidelity, and speculative hit-rate optimization.
• Contribute to experimental designs for novel training regimes and speculative decoding strategies.

Ideal candidates have/are:

• 5+ years of experience in machine learning, with a strong focus on model training.
• Proven experience with transformer-based architectures (e.g., LLaMA, Mistral, Gemma).
• Deep understanding of speculative decoding and draft model usage.
• Hands-on experience with quantization-aware training, including PyTorch QAT workflows or similar frameworks.
• Familiarity with open-weight foundation models and continued/pre-training techniques.
• Proficient in Python and ML frameworks such as PyTorch, JAX, or TensorFlow.

Preferred Qualifications:

• Experience optimizing models for fast inference and sampling in production environments.
• Exposure to distributed training, low-level kernel optimizations, and inference-time system constraints.
• Publications or contributions to open-source ML projects.

Attributes of a Groqster:

• Humility - Egos are checked at the door
• Collaborative & Team Savvy - We make up the smartest person in the room, together
• Growth & Giver Mindset - Learn it all versus know it all, we share knowledge generously
• Curious & Innovative - Take a creative approach to projects, problems, and design
• Passion, Grit, & Boldness - no limit thinking, fueling informed risk taking

If this sounds like you, we’d love to hear from you!

Compensation: At Groq, a competitive base salary is part of our comprehensive compensation package, which includes equity and benefits. For this role, the base salary range is $175,900 to $307,800, determined by your skills, qualifications, experience and internal benchmarks.