Staff Technical Product Manager – AI/LLM expertise + AI Evaluation Science

About the role

We are looking for a highly technical, strategic, and outcome-oriented Staff Technical Product Manager to establish and scale AI Evaluation as a core engineering capability across the DT organization. The Eval Technical Product Manager will lead this transformation by making trusted Evals the evidence layer for AI.

What you'll do

• Identify gaps where OOTB Evals don't adequately measure enterprise AI quality
• Define and drive custom metrics, rubrics, datasets and evaluation strategies
• Develop reusable Evals for domain-specific behaviors and business outcomes
• Establish evaluation approaches for AI agents, task completion, tool usage and multi-step workflows
• Establish evaluation coverage and quality thresholds

What you need

• 10–18 yrs of experience
• LLMs
• ServiceNow
• CI/CD
• Any graduate