Structured Output

When structure matters

A person can read "the priority is probably 2, but it might be higher if the radio link fails too". A program cannot. Structured output means making a model answer in a fixed, machine-readable format, usually JSON, so that code can take the answer and put it in a ticket field, a database or a report. Structure helps people too: a table of affected sites, or a status update that always has the same three headings, is faster to read and to compare than free prose.

There are three levels of control, from weakest to strongest:

1. Asking in the prompt. Works with every assistant; the model usually follows it, but nothing enforces it.
2. JSON mode. An API setting that guarantees valid JSON, but not which fields it contains.
3. Schema enforcement. An API feature that makes the answer match a schema you supply.

Whatever the level, the program that receives the answer still checks it. The rest of this page goes through the levels and then through that check.

Describing the format in the prompt

A model can only produce the format you describe, and "as JSON" describes very little. Which keys? What type is a priority: 2, "2", "P2" or "high"? What if the email does not mention a site? Every question you leave open is answered by the model, and possibly differently on each run.

Give me the important info from this email as JSON.

Return only a JSON object, with no text before or after it and no code fence, with exactly these keys:
- "ticket": the ticket number, a string like "INC-12345"
- "site": one of "Utrecht", "Zwolle", "Groningen"
- "priority": an integer from 1 (highest) to 4, only if the email states the current priority
- "summary": at most 15 words
Use null for any value the email does not give. Do not guess.

<email>
(the email text)
</email>

The specific prompt answers each open question in advance: the keys, the types, the allowed values, the length, and what to do with missing data. That last point matters most. Without an explicit "use null", the most plausible continuation of a JSON object with a priority key is a number, so the model may invent one.

For output that people read, the same principle applies to tables and fixed text layouts. Name the columns or headings and their order; Microsoft's guidance also notes that models read tables well and that tables use fewer tokens than the same data as JSON.

Write the status update with exactly these three headings, in this order, each followed by one or two sentences: Impact, Cause, Next update. If the cause is not known yet, write "Under investigation" under Cause.

Give me the important info from this email as JSON.

From: Tomasz Nowak (NOC)
Subject: link down Zwolle
Hi, the fiber link between the Zwolle substation and the NOC dropped at 06:40 this morning. We're on the backup radio link now, so measurements still come in, just slower. Ticket is INC-40388. Not urgent unless the radio link also goes, then it's a P1. Will update after the contractor has been. Tomasz

Return only a JSON object, with no text before or after it and no code fence, with exactly these keys:
- "ticket": the ticket number, a string like "INC-12345"
- "site": one of "Utrecht", "Zwolle", "Groningen"
- "priority": an integer from 1 (highest) to 4, only if the email states the current priority
- "summary": at most 15 words
Use null for any value the email does not give. Do not guess.

From: Tomasz Nowak (NOC)
Subject: link down Zwolle
Hi, the fiber link between the Zwolle substation and the NOC dropped at 06:40 this morning. We're on the backup radio link now, so measurements still come in, just slower. Ticket is INC-40388. Not urgent unless the radio link also goes, then it's a P1. Will update after the contractor has been. Tomasz

Answers vary between runs and between assistants, which is exactly the problem structured output in applications has to deal with.

JSON mode and schema enforcement

In an API you can go beyond asking. The weaker option is JSON mode: an API setting that guarantees the answer is syntactically valid JSON, but not that it has the keys, types or values you want. OpenAI's documentation states the difference plainly: JSON mode ensures valid JSON, and only its Structured Outputs feature ensures that the answer adheres to your schema.

Schema adherence is the property that an answer matches a given schema: all required fields present, every value of the right type, only allowed values and no extra fields. The schema is usually written in JSON Schema:

{
  "type": "object",
  "properties": {
    "ticket":   { "type": "string", "pattern": "^INC-[0-9]{5}$" },
    "site":     { "type": ["string", "null"], "enum": ["Utrecht", "Zwolle", "Groningen", null] },
    "priority": { "type": ["integer", "null"], "minimum": 1, "maximum": 4 },
    "summary":  { "type": "string" }
  },
  "required": ["ticket", "site", "priority", "summary"],
  "additionalProperties": false
}

Providers enforce a schema with constrained decoding: while the model generates the answer token by token, only tokens that keep the output valid according to the schema are allowed. The model cannot write a stray sentence before the JSON or a fifth key, because those tokens are never permitted. That is why a complete enforced answer parses (refusals and cut-off answers are the exceptions, see below), and also why it cannot fix content: the constraint decides which tokens are possible, not which value is true.

As an example of the request side, this is the relevant part of a request to OpenAI's Responses API with Structured Outputs, as documented in October 2026 (the schema goes where the placeholder is):

{
  "model": "MODEL_ID",
  "input": [
    { "role": "user", "content": "Extract the ticket fields from this email: ..." }
  ],
  "text": {
    "format": {
      "type": "json_schema",
      "name": "ticket_fields",
      "strict": true,
      "schema": { "type": "object", "properties": {} }
    }
  }
}

What the providers offer

All three large API providers support schema-based output, each with its own limits (as of October 2026):

The schema above uses minimum and maximum. A provider that does not support those keywords cannot enforce the 1 to 4 range, so your own validation has to. That is the general pattern: the provider's guarantee covers part of your rules, and your code covers the rest.

Strict tool use

Structure also matters when a model calls tools (see tool calling): each tool has an input schema, and the model generates the arguments. Strict tool use is a setting on a tool definition (strict: true at both Anthropic and OpenAI, as of October 2026) that makes the provider enforce the tool's input schema, so the arguments of every call match it. OpenAI recommends always enabling it. For a classification, a tool with a single enum field (for example category with the values Account, Billing, Network, Portal) is a clean way to get exactly one allowed label back.

Strict mode makes the arguments well formed; whether calling the tool with them is a good idea is a separate question. Tool calls that change something still need permission checks and, for risky actions, human approval.

Checking the answer in code

An output parser is the code that turns a model's raw text answer into data a program can use: it extracts the relevant part, parses it, validates it against the expected shape, and rejects or retries what does not fit. Every application that uses model output needs one, with or without schema enforcement, for the same reason any program validates input from outside: the answer is untrusted until checked.

Extracting the JSON

Without enforcement, models often wrap JSON in friendly text or in a fenced code block. The parser below prefers the content of a fence when there is one, and otherwise tries the whole answer. It returns either data or a reason, so the caller can decide what to do; it does not crash and does not guess.

import json
import re

# A fenced block: three backticks, optionally "json", the content, three backticks.
FENCE = re.compile(r"```(?:json)?\s*\n(.*?)\n\s*```", re.DOTALL)

# Four recorded model answers: text, not yet data.
answers = {
    "plain": '{"ticket": "INC-40388", "site": "Zwolle", "priority": 2}',
    "fenced": ("Here is the data you asked for:\n"
               "```json\n"
               '{"ticket": "INC-40388", "site": "Zwolle", "priority": 2}\n'
               "```\n"
               "Let me know if you need anything else."),
    "trailing comma": ('```json\n'
                       '{"ticket": "INC-40388", "site": "Zwolle",}\n'
                       '```'),
    "no JSON": "I could not find a ticket number in this email.",
}


def parse_answer(text):
    """Return (data, None) or (None, reason). Prefer a fenced block if present."""
    match = FENCE.search(text)
    candidate = match.group(1) if match else text.strip()
    try:
        return json.loads(candidate), None
    except json.JSONDecodeError as err:
        return None, f"not valid JSON: {err.msg} (column {err.colno})"


for name, text in answers.items():
    data, error = parse_answer(text)
    print(f"{name:<15} {error or data}")

plain           {'ticket': 'INC-40388', 'site': 'Zwolle', 'priority': 2}
fenced          {'ticket': 'INC-40388', 'site': 'Zwolle', 'priority': 2}
trailing comma  not valid JSON: Expecting property name enclosed in double quotes (column 42)
no JSON         not valid JSON: Expecting value (column 1)

The pattern is a regular expression with re.DOTALL, so .*? also matches line breaks, and the ? makes it stop at the first closing fence. Do not try to repair broken JSON by deleting commas or adding quotes: a repaired answer is a guess, and a retry is cheap.

Validating and normalizing the fields

Parsing proves only that the text is JSON. The next step checks the shape and the values against your own rules, collects every problem rather than stopping at the first, and normalizes harmless differences. Anthropic's documentation warns that the letter case of enum values is not guaranteed, so "ZWOLLE" is accepted and stored as "Zwolle", while "Amsterdam" is rejected.

import re

SITES = {"utrecht": "Utrecht", "zwolle": "Zwolle", "groningen": "Groningen"}
KEYS = {"ticket", "site", "priority", "summary"}
TICKET = re.compile(r"INC-\d{5}")


def validate(data):
    """Check one parsed answer; fix letter case of the site; list all problems."""
    problems = []
    if extra := sorted(set(data) - KEYS):
        problems.append(f"unexpected keys {extra}")
    if missing := sorted(KEYS - set(data)):
        problems.append(f"missing keys {missing}")

    ticket = data.get("ticket")
    if not (isinstance(ticket, str) and TICKET.fullmatch(ticket)):
        problems.append(f"ticket {ticket!r} is not like INC-12345")

    site = data.get("site")  # null (None) is allowed: "not in the email"
    if isinstance(site, str) and site.lower() in SITES:
        data["site"] = SITES[site.lower()]
    elif site is not None:
        problems.append(f"unknown site {site!r}")

    priority = data.get("priority")  # type() is int: rejects True and 2.0
    if priority is not None and not (type(priority) is int and 1 <= priority <= 4):
        problems.append(f"priority {priority!r} is not 1-4")

    summary = data.get("summary")
    if not isinstance(summary, str) or len(summary.split()) > 15:
        problems.append("summary missing or longer than 15 words")
    return problems


# Three recorded answers, already parsed from JSON.
parsed = [
    {"ticket": "INC-40388", "site": "ZWOLLE", "priority": 2,
     "summary": "Substation link down since 06:40, NOC investigating"},
    {"ticket": "INC-40391", "site": "Amsterdam", "priority": "high",
     "summary": "Meter data delayed"},
    {"ticket": "40392", "site": None, "priority": None,
     "summary": "Portal login slow", "confidence": 0.9},
]

for data in parsed:
    problems = validate(data)
    if problems:
        print("REJECT", "; ".join(problems))
    else:
        print("ACCEPT", data)

ACCEPT {'ticket': 'INC-40388', 'site': 'Zwolle', 'priority': 2, 'summary': 'Substation link down since 06:40, NOC investigating'}
REJECT unknown site 'Amsterdam'; priority 'high' is not 1-4
REJECT unexpected keys ['confidence']; ticket '40392' is not like INC-12345

In production, a JSON Schema library can do most of these checks from the schema itself, so the schema you send to the provider and the one you validate against are the same file. The hand-written version shows what such a check has to cover; the structured output section of Working with LLMs shows another, including why True slips through a plain integer check in Python.

Refusals, cut-off answers and retries

Even with schema enforcement, some responses contain no usable answer, and the API tells you why in the response metadata, not in the text:

• Refusal. The model declined the request. Both Anthropic and OpenAI document that a refusal may not follow the schema; OpenAI returns it in a separate refusal field, Anthropic as a stop reason. Retrying the same request rarely helps; a person should look at it.
• Cut off. The answer reached the maximum output length and stopped mid-object. On models that reason before answering, the reasoning counts toward that limit too. Retry with a higher limit or a shorter task (see Prompts in Applications: the API).
• Invalid content. Without enforcement: not JSON, wrong keys, wrong values. One retry, possibly with the validation error added to the prompt, often fixes it.

The example runs three recorded scenarios through a small decision function and a retry loop with a fixed maximum. The stop reasons follow the style of Anthropic's Messages API; the logic is the same for any provider.

import json

MAX_ATTEMPTS = 2


def check(response):
    """Decide what to do with one response: ('accept', data) or (action, reason)."""
    # Stop reasons in the style of Anthropic's Messages API (October 2026);
    # other APIs report the same situations under other names.
    if response["stop_reason"] == "refusal":
        return "to a person", "the model declined the request"
    if response["stop_reason"] == "max_tokens":
        return "retry", "answer cut off at the output limit"
    try:
        return "accept", json.loads(response["text"])
    except json.JSONDecodeError as err:
        return "retry", f"not valid JSON ({err.msg})"


def run(name, recorded_responses):
    """Try up to MAX_ATTEMPTS recorded responses, then fall back to a person."""
    print(f"--- {name}")
    for attempt, response in enumerate(recorded_responses[:MAX_ATTEMPTS], start=1):
        action, detail = check(response)
        print(f"attempt {attempt}: {action}: {detail}")
        if action != "retry":
            return
    print("giving up: ticket goes to the manual triage queue")


run("cut off, then complete", [
    {"stop_reason": "max_tokens", "text": '{"ticket": "INC-40388", "site": "Zwo'},
    {"stop_reason": "end_turn", "text": '{"ticket": "INC-40388", "site": "Zwolle"}'},
])
run("refused", [
    {"stop_reason": "refusal", "text": ""},
])
run("invalid twice", [
    {"stop_reason": "end_turn", "text": "Sure! The ticket is INC-40388."},
    {"stop_reason": "end_turn", "text": "{'ticket': 'INC-40388'}"},
])

--- cut off, then complete
attempt 1: retry: answer cut off at the output limit
attempt 2: accept: {'ticket': 'INC-40388', 'site': 'Zwolle'}
--- refused
attempt 1: to a person: the model declined the request
--- invalid twice
attempt 1: retry: not valid JSON (Expecting value)
attempt 2: retry: not valid JSON (Expecting property name enclosed in double quotes)
giving up: ticket goes to the manual triage queue

Two design choices matter here. The number of attempts is fixed, so a persistent problem costs two requests rather than an endless loop. And the fallback is a person, not a default value: a ticket silently filed under a default category is harder to find than one in a manual queue. Log every rejected answer as well; a sudden rise in rejections is often the first sign that a model update or a prompt change broke something (see Iterating and Testing Prompts).

Advice that has aged

Older tutorials contain format tricks that no longer apply or were never guarantees:

• "Prefill the answer with {." Starting the model's reply yourself used to be a common way to force JSON with Claude. As of October 2026, Claude models from the 4.6 generation on reject a prefilled final assistant turn with an error; Anthropic points to structured outputs instead.
• "JSON mode means the data is right." JSON mode guarantees syntax, schema enforcement guarantees shape, and neither guarantees content.
• "Write FORMAT IN CAPITALS so the model obeys." A precise description and an example of the format do more than emphasis; where the API offers schema enforcement, use it.