Build a Small Eval Before You Choose an AI Model: Anthropic's Test Design Rules in Practice
The most reliable way to choose an AI model for your product is to test candidates on your own tasks with a small eval you build yourself. A public benchmark score tells you how a model did on someone else's questions. An eval tells you how it does on yours. Anthropic's developer documentation sets out how to write success criteria and design tests, and this guide turns its rules into a plan a small team can finish in a week. The guidance was read on 10 November 2026.
Key facts
- Anthropic says good success criteria are specific, measurable, achievable and relevant.
- Its first design principle: make evals task-specific, mirroring your real-world task distribution and including edge cases.
- Its second: automate grading where possible, through multiple choice, string match, code grading or LLM grading.
- Its third: prefer more questions with slightly lower signal and automated grading over fewer, high-quality questions graded by hand.
- For LLM-based grading, the page advises using a different model from the one that produced the output being evaluated.
Step 1: write the success criteria first
The page gives a plain example. Instead of the goal "good performance", write "accurate sentiment classification". The criterion must be something you can score. It must also be achievable, which Anthropic describes as consistent with industry benchmarks, prior experiments and what current frontier models can do, and relevant to the application. It gives the example that strong citation accuracy might be critical for a medical application but matter less for a casual chatbot.
For a support assistant, a criterion could be: the answer matches the policy document on refund eligibility. For a summarizer: the summary contains the three decisions in the source and states no fact that is not in it.
Step 2: collect cases from real work
Anthropic's advice is to mirror the real distribution of your tasks, edge cases included. Pull cases from tickets, emails or documents your team already handles, and remove personal data before you send anything to a vendor. Keep a few cases that you know are hard or unusual, because those are where models differ most.
This is our suggestion, not a figure from Anthropic: start with enough cases that one odd result does not decide the outcome, and grow the set each time a model fails in production.
Step 3: choose a grader for each criterion
The Anthropic page shows four grading approaches in its examples.
- Exact match in code: normalize the output and compare it with a known answer. It suits classification and extraction.
- Similarity metrics in code: embedding similarity for consistency, and ROUGE-L for summaries.
- LLM grading: a model rates the output on a scale, for example 1 to 5 for tone, or answers a yes or no question, such as whether personal health information appears.
- Human grading: slow and hard to scale, so use it to check a sample of the automated scores.
Anthropic's third principle favors volume. A large set graded automatically, with some noise, beats a small set graded by hand with care. Use the human pass to confirm that your automatic grader agrees with a person on a sample.
Step 4: run every candidate the same way
Fix the prompt, the settings and the cases, then change only the model. Record the score, the cost per case and the response time. Pricing differs between vendors in more than the headline rate, as our guide to reading LLM API pricing explains, so compute the cost from your logged tokens, not from the price table.
Step 5: keep the eval and re-run it
An eval is an asset. Every model upgrade or vendor deprecation, such as the ones we track in our OpenAI deprecation calendar, becomes a one-day job instead of a debate. Add each production failure as a new case, so the eval grows with your product.
Why public benchmarks are not enough
Benchmarks measure a fixed set of tasks under the lab's own setup. Your tasks have different inputs, different tolerance for error and different cost limits. Our analysis of benchmark contamination shows how far a headline score can drift from what it claims to measure. Use public scores to build a shortlist, then decide with your own eval. To find candidate products to test, start from the AI tool directory.
What to do this week
- Write three success criteria that a person could score without arguing.
- Pull real cases, anonymize them and include the awkward ones.
- Pick code grading where the answer is checkable, and a different model as an LLM grader for the rest.
- Run two or three candidate models with identical prompts and settings.
- Compare score, cost per case and latency, then keep the eval for the next change.
Questions people ask
What makes a good success criterion for an AI task?
Anthropic says it should be specific, measurable, achievable and relevant. "Accurate sentiment classification" is specific where "good performance" is not.
Should I grade with another model?
Anthropic's page says it is generally best practice to evaluate with a different model from the one that produced the output being evaluated.
Is it better to have fewer hand-graded cases or more automated ones?
Anthropic's guidance prefers more cases with slightly lower signal and automated grading over fewer, high-quality, hand-graded cases.
Sources