Accounting Expert: AI Model Evaluation

About the role

You will create prompts that are difficult enough for a frontier model to fail, then write a scoring rubric for each task so reviewers can assess the output consistently. A rubric is a list of objective, checkable criteria that a reviewer marks pass or fail. Your judgement helps define the standards used to measure model performance.

What you'll do

• Write realistic accounting tasks: Create requests of the kind a client, controller or business owner would send, covering month-end close
• Build from real material: Review a deal room containing hundreds of files
• Test against a frontier model: Run each request and review the output with the care you would apply when signing off accounting work. Where the model succeeds
• Write scoring rubrics: Define objective criteria that another reviewer can apply without repeating your work, for example
• Apply guidelines consistently: Follow the task and rubric guidelines, flag ambiguous cases and help refine the criteria

What you need

• Accounting
• Reconciliation
• Month-end close
• Costing
• Bookkeeping