AI Infrastructure Engineer – Generative AI Model Serving (Go / Rust / GPU)

About the role

AI Infrastructure Engineer – Generative AI Model Serving (Go / Rust / GPU) at Teradata. Design, build, and operate scalable, low-latency model-serving platforms for LLMs and generative models, covering batching, streaming, routing, autoscaling.

What you'll do

• Design, build, and operate scalable, low-latency model-serving platforms for LLMs and generative models, covering batching, streaming, routing, autoscaling
• Develop high-performance control-plane and data-plane services in Go and performance-critical components (tokenizers, schedulers, runtimes, gRPC/HTTP gateways
• Optimize GPU inference for throughput, latency, and cost using continuous batching, KV-cache management, quantization (INT8/FP8/AWQ/GPTQ), speculative decoding
• Integrate and extend inference engines such as vLLM, TensorRT-LLM, Triton Inference Server, Ray Serve, KServe, SGLang, llama.cpp, and ONNX Runtime
• Build and manage GPU cluster infrastructure on Kubernetes (NVIDIA GPU Operator, device plugins, MIG/time-slicing, node pools

What you need

• 3–6 years in data science & ai
• LLMs
• Go
• Rust
• Machine learning
• Python
• PyTorch
• B.Tech or B.E. or B.Sc or BCA or M.Tech or MCA or M.Sc

Good to have

• Snowflake
• Databricks
• Technical writing
• gRPC
• CI/CD
• Terraform