MTS 2, AI Platform

About the role

As an LLM Inference Engineer on our AI Platform team, you'll remove the compute-scaling bottleneck for production LLMs. Your job is to make frontier-model inference fast, efficient, reliable, and observable — the "last mile" from GPUs to APIs that products depend on. This role sits at the intersection of HPC, GPU systems, and MLOps, and requires strong intuition for how model architecture, runtimes, and hardware interact.

What you'll do

• Own production inference: take models from handoff to production-grade serving, including release engineering, capacity planning, cost optimization, and incident response
• Optimize runtimes and servers: scale inference across heterogeneous GPU fleets; optimize stacks such as vLLM, Triton, and related components (e.g., schedulers, KV cache
• Apply and ship new optimizations: evaluate research and implement pragmatic inference optimizations (e.g., quantization, paging, kernel/runtime improvements)
• Partner with cross-functional teams: work with data science and product teams to translate business requirements into performance and availability SLOs

What you need

• 8–14 yrs of experience
• Machine learning
• LLMs
• Python
• PyTorch
• Root cause analysis
• Go
• Any graduate

Good to have

• Incident response
• MLOps
• Last-mile delivery