Senior Technical Lead

About the role

The AI Quality Engineer (AI QE) is responsible for validating the quality, reliability, safety, and user experience of AI and Generative AI solutions. The role combines software quality engineering with AI evaluation to test Large Language Model (LLM) applications, copilots, chatbots, Retrieval-Augmented Generation (RAG) systems, and agentic workflows across real-world user scenarios.

What you'll do

• Validate AI-powered features, LLM applications, copilots, chatbots, agents, and conversational experiences across functional and non-functional requirements
• Design test strategies for deterministic application behavior and probabilistic AI outputs, covering positive, negative, adversarial, boundary, and edge-case scenarios
• Test agent behavior across user personas, multi-step workflows, tool usage, function calling, memory, routing, fallback, and error-recovery paths
• Evaluate AI-generated responses across single-turn and multi-turn conversations for factual correctness, grounding, relevance, completeness, coherence
• Create and maintain ground-truth, golden, and benchmark datasets for evaluation and regression testing

What you need

• 4–8 yrs of experience
• Python
• Unit testing
• AWS
• LLMs
• CI/CD
• Machine learning
• Any graduate

Good to have

• Azure
• Bug tracking
• Prompt engineering
• API testing
• Test automation
• Usability testing