AI DevOps Engineer

About the role

We are seeking a highly motivated AI DevOps Engineer to join Deltek’s growing AI Center of Excellence.The ideal candidate combines deep expertise in LLMOps, MLOps, and DevOps with hands-on experience in Generative AI, Large Language Models (LLMs), SLMs, and cloud computing.

What you'll do

• LLMOps & Generative AI DevOps
• Design and operate LLMOps pipelines for prompt management, evaluation, versioning, release, and rollback of LLM-based applications
• Run and scale LLM gateways and model-serving layers across providers such as GPT, Claude, Gemini, Llama, and Mistral, including routing, rate limiting, caching
• Operate Retrieval-Augmented Generation (RAG) platforms, including vector database management, embedding refresh, and index lifecycle
• Build automated evaluation frameworks to measure quality, groundedness, hallucination rates, safety, latency, and cost per request

What you need

• CI/CD
• Docker
• Kubernetes
• Python
• LLMs
• MLOps

Good to have

• Terraform
• AWS
• GCP
• Azure
• Incident response
• Root cause analysis