← All Articles
Sign in
Developer Tools Blog

Structured Outputs vs Function Calling vs JSON Mode: Which One to Use and What Breaks

By AI SaaS Radar Team · Nov 2026 · 5 min read

Use structured outputs when you want the model's reply to follow a schema, use function calling when the model must call your tools, and avoid plain JSON mode when a schema-enforcing option exists. All three return JSON, but only structured outputs and strict tool use promise that the JSON matches your schema. Both OpenAI and Anthropic still document exceptions, namely refusals and truncated answers, so your code needs a path for them. This guide summarizes both vendors' documentation as read on 10 November 2026.

Key facts

Pick by what you are building

The vendors draw the same line. If the model's job is to act, for example look up an order or book a slot, define tools and use function calling. If its job is to answer in a fixed shape, for example extract fields from an invoice or classify a ticket, use a schema on the response. Anthropic puts it as the difference between controlling what the model says and validating how it calls your functions. You can combine them: Anthropic notes a single request can call tools with guaranteed-valid parameters and also return a structured JSON response.

Why JSON mode is the weak option

JSON mode guarantees syntax, not shape. A reply can parse and still miss a required field, use the wrong type or invent a key. OpenAI recommends Structured Outputs over JSON mode wherever possible, and Anthropic's feature is described as removing JSON parse errors and schema-violation retries. If you are still validating and retrying in a loop, check whether a schema-enforcing option now replaces that code.

Schema limits to check before you commit

Enforced schemas support a subset of JSON Schema. Anthropic's page lists what is supported and what is not:

OpenAI's raw JSON Schema examples use required on every object and additionalProperties false. Test your real schema against each vendor you might use, because a schema that works on one may be rejected by the other.

The two cases where the schema is not guaranteed

First, refusals. OpenAI says a refusal does not necessarily follow your schema, and tells you to check the refusal field. Anthropic says a refusal returns stop_reason refusal with a 200 status, you are billed for the tokens, and the output might not match the schema. Second, truncation. OpenAI says a response with status incomplete and reason max_output_tokens may be cut off. Anthropic says stop_reason max_tokens can leave the output incomplete, and advises a retry with a higher limit. Treat both as expected states, not exceptions.

Latency, caching and token cost

OpenAI says the first request with any schema has additional latency while the schema is processed, and later requests with the same schema have none. Anthropic describes the same effect as grammar compilation, cached for 24 hours from last use, and invalidated when the schema structure changes. Anthropic adds two costs. Changing output_config.format invalidates the prompt cache for that conversation thread, and the model receives an extra system prompt describing the format, so input token counts rise slightly. If you change schemas often, expect more cache misses.

What to do

Agent builders may also want our overview of computer-use agent architectures, and a comparison of Claude Code, Cursor and Copilot. To find developer tools, browse the AI tool directory.

Questions people ask

What is the difference between JSON mode and structured outputs?

JSON mode returns valid JSON without enforcing your schema. Structured outputs enforce the schema you supply. OpenAI recommends structured outputs over JSON mode when possible.

Does a schema guarantee valid output every time?

Not in two cases. A refusal may not follow the schema, and an answer cut off by the token limit can be incomplete. Both vendors document these exceptions.

Can I use function calling and a response schema together?

Anthropic says yes: JSON outputs and strict tool use can be used independently or in the same request.

Sources

Stay ahead of the AI SaaS market

Sourced, dated analysis on security, funding, and benchmarks. Straight to your inbox.

No spam. Unsubscribe anytime.