Staff Software Development Test Engineer - AI Evaluation

About the role

We are looking for a highly motivated Staff SDET – AI Evaluation to join Tekion’s AI Platform team. Evaluation is the backbone of trustworthy AI: as Tekion scales from a handful of AI agents

What you'll do

• Create, curate, and maintain evaluation datasets (evals) and golden/ground-truth sets across use cases and domains
• Define quality metrics for AI outputs — accuracy, relevance, faithfulness/groundedness, consistency, safety, and task success
• Build automated scoring pipelines, including LLM-as-judge, rubric-based, and referencebased evaluation methods
• Validate user intents and measure response accuracy and consistency for AI-powered capabilities such as the Analytics Agent
• Identify hallucinations, unsafe or biased outputs, and edge cases; design targeted eval suites to catch them

What you need

• 8–14 years in software engineering
• Python
• Machine learning
• LLMs
• CI/CD
• Any graduate

Good to have

• MLOps