← All Articles
Sign in
Developer Tools Blog

Batch API vs Real-Time Calls: When the 50% Discount Fits and What You Give Up

By AI SaaS Radar Team · Nov 2026 · 5 min read

Batch APIs from Anthropic and OpenAI charge 50 percent less than real-time calls, in exchange for waiting. If a job does not need an answer in seconds, such as classifying an archive, scoring an evaluation set or enriching a database overnight, a batch halves the model bill. If a person is waiting on the answer, it does not fit. This guide lists the limits, the data-retention catch and a worked cost example, all from the vendors' documentation as read on 10 November 2026.

Key facts

What you give up

A batch is asynchronous. You submit a file or list of requests, poll for status and download results when the job completes. Both vendors set the outer limit at 24 hours. Anthropic adds that most batches complete within an hour, and that processing might be slowed by demand and your request volume, in which case more requests can expire after 24 hours. Plan for the 24-hour case, not the average one.

Every request in a batch is independent. You cannot run a conversation across turns, and you cannot use the answer to one request to shape the next within the same batch. Anthropic also states that fast mode is not available with the Batch API.

The limits that shape your job size

Anthropic allows 100,000 requests or 256 MB per batch. OpenAI allows 50,000 requests and 200 MB per input file, and also caps queued prompt tokens per model and 2,000 batches per hour. A job of 100,000 items therefore fits in one Anthropic batch and needs at least two OpenAI batches. Neither limit is a problem for most teams, but build the splitting logic before you need it.

OpenAI's batch endpoints include responses, chat completions, embeddings, completions, moderations and image generation and edits. Anthropic's covers Messages requests, with a long list of supported Claude models. Check that your exact model and feature appear in the current docs before you design a pipeline around it.

A worked example

This is our own arithmetic from published list prices. Anthropic lists Claude Sonnet 5.5 at $2 per million input tokens and $10 per million output tokens, and at $1 and $5 for batch. Suppose you classify 100,000 support tickets, each with 800 tokens of input and 100 tokens of output.

The saving is $130 for one run. A nightly job of this size saves about the same each night, which is why batch is the first option to check for any recurring back-office workload.

Combining batch with caching

If every request in the batch starts with the same long instructions, prompt caching can cut the input cost further. Anthropic's pricing page says its cache multipliers stack with the batch discount. Its batch documentation adds a warning: because batch requests run asynchronously and concurrently, cache hits are provided on a best-effort basis. It advises identical cache settings across the batch and the 1-hour cache duration, since a batch can take longer than 5 minutes. Treat any caching saving inside a batch as a bonus, not a number to promise.

The data-retention catch

Batch jobs store your inputs and results so you can download them later. Anthropic's batch page lists the feature as not eligible for zero data retention and gives a 29-day retention note. OpenAI's data controls page lists Batches among endpoints not eligible for zero data retention. If your contract or your customers require zero data retention, a batch is the wrong tool for that data, whatever the discount.

What to do

For the vendor side of these prices, see how token prices and vendor margins are moving. Developers can compare coding and API tools in the AI tool directory.

Questions people ask

How long does a batch take?

Anthropic says most batches finish within 1 hour, and a batch expires if it is not done within 24 hours. OpenAI's completion window is 24 hours, and batches often finish sooner.

Does a batch use my normal rate limit?

On OpenAI, no: batch has its own rate-limit pool, separate from per-model limits. On Anthropic, rate limits apply to the Batches API requests and to the number of requests waiting to be processed.

Can I use batch with zero data retention?

Neither vendor lists batch as eligible. Both Anthropic's batch documentation and OpenAI's data controls page mark it as not eligible, so check your agreement before sending regulated data.

Sources

Stay ahead of the AI SaaS market

Sourced, dated analysis on security, funding, and benchmarks. Straight to your inbox.

No spam. Unsubscribe anytime.