Build and optimize scalable pre-training pipelines for foundation models. Implement distributed training across GPUs, TPUs, and high-performance clusters; translate research experiments into production workflows; develop monitoring and fault-tolerance systems; and benchmark and tune performance across hardware and software stacks.