Senior ML Engineer - Inference and Model Serving

About the role

We are hiring a senior ML engineer to own how our language and speech models run in production. agents depend on fast, reliable inference: every spoken reply passes through speech recognition, a language model and speech synthesis within a tight latency budget. You will serve these models on GPUs at the lowest cost per call without losing quality, build the security and metering layers around

What you'll do

• Serve open-weight language models with engines such as vLLM, TensorRT-LLM or SGLang, tuned
• For both time to first token and throughput
• Apply continuous batching, prefix caching, speculative decoding and multi-LoRA serving to cut
• Latency and cost per request
• Manage GPU memory with PagedAttention, FlashAttention, KV-cache quantisation and tensor

What you need

• 4–8 yrs of experience
• Machine learning
• Python
• LLMs
• MLOps
• PyTorch
• CI/CD
• B.Tech or B.E. or B.Sc or BCA or PhD

Good to have

• Kafka
• Kubernetes
• C++
• Prometheus