Senior QA Engineer, AI Agents & Agentic Workflows
PuneFull-time3–5 yrsPosted yesterday
Mid-level QA and testing role at QAD in Pune. Needs 3–5 years and a B.Tech, B.E. or B.Sc.
About the role
agent AI product with ~20 evolving agents. You will test the agents themselves: their reasoning, tool use, orchestration and handoffs, and the quality of their responses. This goes beyond UI and API checks. You will design evaluation frameworks and regression suites that catch failures unique to non-deterministic, LLM-
What you'll do
- Design, run and maintain end-to-end business scenarios that exercise multi-step agentic workflows
- Including agent routing, tool/function calls, and multi-turn conversations
- Evaluate agent outputs for correctness, relevance, groundedness, consistency, tone and
- Completeness, using both human review and automated methods
- Build and maintain eval datasets (golden sets, edge cases, adversarial prompts) and automated
What you need
- API testing
- Selenium
- Test automation
- Playwright
- LLMs