Batch API vs Real-Time Calls: When the 50% Discount Fits and What You Give Up
Batch APIs from Anthropic and OpenAI charge 50 percent less than real-time calls, in exchange for waiting. If a job does not need an answer in seconds, such as classifying an archive, scoring an evaluation set or enriching a database overnight, a batch halves the model bill. If a person is waiting on the answer, it does not fit. This guide lists the limits, the data-retention catch and a worked cost example, all from the vendors' documentation as read on 10 November 2026.
Key facts
- Anthropic's Message Batches API charges 50 percent of standard prices on input and output tokens. A batch holds up to 100,000 requests or 256 MB, whichever comes first.
- Anthropic says most batches finish within 1 hour, and a batch expires if processing is not complete within 24 hours. Results stay available for 29 days after creation.
- OpenAI's Batch API is 50 percent cheaper than synchronous calls, with a 24-hour completion window that can only be set to 24h. A batch file holds up to 50,000 requests and 200 MB.
- OpenAI gives batch its own rate-limit pool, so batch jobs do not use up your standard per-model limits.
- Anthropic's batch documentation marks the feature as not eligible for zero data retention, and OpenAI's data controls page lists Batches among the endpoints not eligible for zero data retention.
What you give up
A batch is asynchronous. You submit a file or list of requests, poll for status and download results when the job completes. Both vendors set the outer limit at 24 hours. Anthropic adds that most batches complete within an hour, and that processing might be slowed by demand and your request volume, in which case more requests can expire after 24 hours. Plan for the 24-hour case, not the average one.
Every request in a batch is independent. You cannot run a conversation across turns, and you cannot use the answer to one request to shape the next within the same batch. Anthropic also states that fast mode is not available with the Batch API.
The limits that shape your job size
Anthropic allows 100,000 requests or 256 MB per batch. OpenAI allows 50,000 requests and 200 MB per input file, and also caps queued prompt tokens per model and 2,000 batches per hour. A job of 100,000 items therefore fits in one Anthropic batch and needs at least two OpenAI batches. Neither limit is a problem for most teams, but build the splitting logic before you need it.
OpenAI's batch endpoints include responses, chat completions, embeddings, completions, moderations and image generation and edits. Anthropic's covers Messages requests, with a long list of supported Claude models. Check that your exact model and feature appear in the current docs before you design a pipeline around it.
A worked example
This is our own arithmetic from published list prices. Anthropic lists Claude Sonnet 5.5 at $2 per million input tokens and $10 per million output tokens, and at $1 and $5 for batch. Suppose you classify 100,000 support tickets, each with 800 tokens of input and 100 tokens of output.
- Real-time: 80 million input tokens at $2 is $160, and 10 million output tokens at $10 is $100. Total $260.
- Batch: the same tokens at $1 and $5 cost $80 and $50. Total $130.
The saving is $130 for one run. A nightly job of this size saves about the same each night, which is why batch is the first option to check for any recurring back-office workload.
Combining batch with caching
If every request in the batch starts with the same long instructions, prompt caching can cut the input cost further. Anthropic's pricing page says its cache multipliers stack with the batch discount. Its batch documentation adds a warning: because batch requests run asynchronously and concurrently, cache hits are provided on a best-effort basis. It advises identical cache settings across the batch and the 1-hour cache duration, since a batch can take longer than 5 minutes. Treat any caching saving inside a batch as a bonus, not a number to promise.
The data-retention catch
Batch jobs store your inputs and results so you can download them later. Anthropic's batch page lists the feature as not eligible for zero data retention and gives a 29-day retention note. OpenAI's data controls page lists Batches among endpoints not eligible for zero data retention. If your contract or your customers require zero data retention, a batch is the wrong tool for that data, whatever the discount.
What to do
- List every job in your product that no user waits for: evaluations, backfills, nightly enrichment, bulk classification, archive summaries.
- Move those to batch and keep interactive paths on real-time calls.
- Build for the 24-hour worst case: retries, partial results and a clear failure alert.
- Split large jobs under the smaller vendor limit, so you can switch vendors without rewriting.
- Check whether the data in the job may be stored for the retention period.
For the vendor side of these prices, see how token prices and vendor margins are moving. Developers can compare coding and API tools in the AI tool directory.
Questions people ask
How long does a batch take?
Anthropic says most batches finish within 1 hour, and a batch expires if it is not done within 24 hours. OpenAI's completion window is 24 hours, and batches often finish sooner.
Does a batch use my normal rate limit?
On OpenAI, no: batch has its own rate-limit pool, separate from per-model limits. On Anthropic, rate limits apply to the Batches API requests and to the number of requests waiting to be processed.
Can I use batch with zero data retention?
Neither vendor lists batch as eligible. Both Anthropic's batch documentation and OpenAI's data controls page mark it as not eligible, so check your agreement before sending regulated data.
Sources