Manager II, Program Management, AI Model Evaluation, International Seller Growth - ISG

About the role

The Seller AI team within International Seller Services is dedicated to creating Gen-AI/LLM powered tools and agentic solutions that accelerate business growth for Amazon sellers worldwide. We focus on handling annotations for training, measuring, and improving Artificial Intelligence and Large Language Models to deliver superior seller experiences.

What you'll do

• Your day begins with prioritizing benchmarking tasks for your team based on current business requirements
• Review progress on ongoing audits, provide guidance where needed, and ensure deliverables meet quality standards
• Throughout the day, you'll analyze performance data from Seller Assistant tools, identifying patterns and insights that could improve AI model effectiveness
• Wrap up by preparing detailed presentations that showcase your team's findings and recommendations to leadership

What you need

• 3–6 yrs of experience
• Project management
• LLMs
• Stakeholder management
• Test cases
• Root cause analysis
• Any graduate

Good to have

• SQL