← All Articles
Sign in
Benchmarks Blog

Build a Small Eval Before You Choose an AI Model: Anthropic's Test Design Rules in Practice

By AI SaaS Radar Team · Nov 2026 · 5 min read

The most reliable way to choose an AI model for your product is to test candidates on your own tasks with a small eval you build yourself. A public benchmark score tells you how a model did on someone else's questions. An eval tells you how it does on yours. Anthropic's developer documentation sets out how to write success criteria and design tests, and this guide turns its rules into a plan a small team can finish in a week. The guidance was read on 10 November 2026.

Key facts

Step 1: write the success criteria first

The page gives a plain example. Instead of the goal "good performance", write "accurate sentiment classification". The criterion must be something you can score. It must also be achievable, which Anthropic describes as consistent with industry benchmarks, prior experiments and what current frontier models can do, and relevant to the application. It gives the example that strong citation accuracy might be critical for a medical application but matter less for a casual chatbot.

For a support assistant, a criterion could be: the answer matches the policy document on refund eligibility. For a summarizer: the summary contains the three decisions in the source and states no fact that is not in it.

Step 2: collect cases from real work

Anthropic's advice is to mirror the real distribution of your tasks, edge cases included. Pull cases from tickets, emails or documents your team already handles, and remove personal data before you send anything to a vendor. Keep a few cases that you know are hard or unusual, because those are where models differ most.

This is our suggestion, not a figure from Anthropic: start with enough cases that one odd result does not decide the outcome, and grow the set each time a model fails in production.

Step 3: choose a grader for each criterion

The Anthropic page shows four grading approaches in its examples.

Anthropic's third principle favors volume. A large set graded automatically, with some noise, beats a small set graded by hand with care. Use the human pass to confirm that your automatic grader agrees with a person on a sample.

Step 4: run every candidate the same way

Fix the prompt, the settings and the cases, then change only the model. Record the score, the cost per case and the response time. Pricing differs between vendors in more than the headline rate, as our guide to reading LLM API pricing explains, so compute the cost from your logged tokens, not from the price table.

Step 5: keep the eval and re-run it

An eval is an asset. Every model upgrade or vendor deprecation, such as the ones we track in our OpenAI deprecation calendar, becomes a one-day job instead of a debate. Add each production failure as a new case, so the eval grows with your product.

Why public benchmarks are not enough

Benchmarks measure a fixed set of tasks under the lab's own setup. Your tasks have different inputs, different tolerance for error and different cost limits. Our analysis of benchmark contamination shows how far a headline score can drift from what it claims to measure. Use public scores to build a shortlist, then decide with your own eval. To find candidate products to test, start from the AI tool directory.

What to do this week

Questions people ask

What makes a good success criterion for an AI task?

Anthropic says it should be specific, measurable, achievable and relevant. "Accurate sentiment classification" is specific where "good performance" is not.

Should I grade with another model?

Anthropic's page says it is generally best practice to evaluate with a different model from the one that produced the output being evaluated.

Is it better to have fewer hand-graded cases or more automated ones?

Anthropic's guidance prefers more cases with slightly lower signal and automated grading over fewer, high-quality, hand-graded cases.

Sources

Stay ahead of the AI SaaS market

Sourced, dated analysis on security, funding, and benchmarks. Straight to your inbox.

No spam. Unsubscribe anytime.