Structured Outputs skill for Claude Code

Structured output is reliable when the provider’s constrained decoding guarantees the shape, your own validator checks the meaning, every failure path (refusal, truncation, semantic violation) is handled explicitly, and an eval measures field-level accuracy. “Respond only in JSON” in a prompt plus a regex is not reliable.

Ground rules

• Use native schema enforcement when the provider supports it. Fall back to prompt-only JSON only for providers or models that lack it, and then always validate and repair.
• Define the schema once in code (Pydantic / Zod) and generate the JSON Schema from it.
• Shape guarantees are not correctness guarantees. Validate values: ranges, cross-field rules, IDs that must exist, totals that must add up.
• Check stop_reason before parsing. Truncated or refused responses may not match the schema.
• Design schemas for the model: flat, descriptive field names, enums for closed sets, explicit null for “not present”.

Workflow

- [ ] 1. Decide the mechanism (output format vs. strict tool vs. prompt-only)
- [ ] 2. Design the schema for extraction quality
- [ ] 3. Implement with the SDK's parse helper
- [ ] 4. Add semantic validation and a bounded repair loop
- [ ] 5. Evaluate field-level accuracy on labeled examples

1. Choose the mechanism

Need	Mechanism
The whole response is data for code	JSON schema output format (output_config.format on the Anthropic Messages API)
The model should decide whether and which action to take, with typed arguments	Tools with strict: true
Both: tools during the turn, typed final answer	Strict tools plus an output format in the same request
Provider/model without constrained decoding	Prompt with schema + examples, then validate and repair
Grounded answers with citations	Native citations instead; on Anthropic these cannot be combined with an output format in one request

Forcing a specific tool through tool_choice used to be the standard extraction trick. Several current Claude models (Opus 5.5, Sonnet 5.5, Fable 5.1) reject forced tool_choice (any / tool) with a 400. Use the output format for extraction, or tool_choice: auto with a strict tool, a prompt instruction naming the tool, and a check that the call happened.

2. Design the schema

from enum import Enum
from pydantic import BaseModel, Field

class Category(str, Enum):
    billing = "billing"
    bug = "bug"
    feature_request = "feature_request"
    other = "other"                      # always include an escape value

class LineItem(BaseModel):
    description: str
    quantity: int
    unit_price_cents: int = Field(description="Price per unit in integer cents, no currency symbol")

class Invoice(BaseModel):
    vendor_name: str
    invoice_number: str | None = Field(description="Exactly as printed; null if absent")
    issue_date: str | None = Field(description="ISO 8601 date YYYY-MM-DD; null if absent")
    currency: str = Field(description="ISO 4217 code such as USD or EUR")
    line_items: list[LineItem]
    total_cents: int
    category: Category

Schema design rules:

• Use null (optional fields) for “not in the source”. Without it the model invents values to satisfy required.
• Enums for every closed set, with an other / unknown member.
• Put units and formats in field names or descriptions (total_cents, ISO dates). Integer cents avoid float rounding.
• Keep nesting shallow (2-3 levels). Deep, wide schemas reduce accuracy and raise latency.
• If you want the model to reason, add a reasoning string field before the answer fields; field order is generation order.
• Avoid unsupported constraints. Anthropic’s structured outputs do not enforce numeric bounds (minimum/maximum), string length, or recursive schemas; the Python and TypeScript SDKs strip those from the wire schema and validate them client-side. Every object must allow no additional properties.

3. Implement

Python (Pydantic, SDK parse helper):

import anthropic

client = anthropic.Anthropic()

resp = client.messages.parse(
    model="claude-sonnet-5-5",
    max_tokens=4096,
    system="Extract invoice data. Use null for any field not present in the document.",
    messages=[{"role": "user", "content": f"<invoice>\n{text}\n</invoice>"}],
    output_format=Invoice,
)
if resp.stop_reason == "refusal":
    raise ExtractionRefused(resp.id)
if resp.stop_reason == "max_tokens":
    raise ExtractionTruncated(resp.id)      # raise max_tokens or split the document
invoice: Invoice = resp.parsed_output

TypeScript (Zod):

import Anthropic from "@anthropic-ai/sdk";
import { z } from "zod";
import { zodOutputFormat } from "@anthropic-ai/sdk/helpers/zod";

const Ticket = z.object({
  category: z.enum(["billing", "bug", "feature_request", "other"]),
  summary: z.string(),
  customer_id: z.string().nullable(),
});

const resp = await client.messages.parse({
  model: "claude-haiku-5-5",
  max_tokens: 1024,
  messages: [{ role: "user", content: `<ticket>${body}</ticket>` }],
  output_config: { format: zodOutputFormat(Ticket) },
});
if (resp.stop_reason !== "end_turn" || !resp.parsed_output) throw new Error(`no output: ${resp.stop_reason}`);
const ticket = resp.parsed_output;

Raw schema without helpers: pass output_config={"format": {"type": "json_schema", "schema": {...}}} to messages.create, then json.loads the text block.

Strict tool definition (arguments guaranteed to validate):

tools = [{
    "name": "create_refund",
    "description": "Issue a refund for an order. Use only when the customer asks for a refund and the order is eligible.",
    "strict": True,
    "input_schema": {
        "type": "object",
        "properties": {
            "order_id": {"type": "string"},
            "amount_cents": {"type": "integer"},
            "reason": {"type": "string", "enum": ["damaged", "late", "not_as_described", "other"]},
        },
        "required": ["order_id", "amount_cents", "reason"],
        "additionalProperties": False,
    },
}]

4. Semantic validation and repair

def extract(text: str, attempts: int = 2) -> Invoice:
    messages = [{"role": "user", "content": f"<invoice>\n{text}\n</invoice>"}]
    for attempt in range(attempts + 1):
        resp = client.messages.parse(model=MODEL, max_tokens=4096, system=SYSTEM,
                                     messages=messages, output_format=Invoice)
        inv = resp.parsed_output
        errors = semantic_errors(inv)          # e.g. sum(line items) != total, bad currency
        if not errors:
            return inv
        messages += [
            {"role": "assistant", "content": resp.content},
            {"role": "user", "content": "Fix these problems and return the full object again:\n- "
                                         + "\n- ".join(errors)},
        ]
    raise ExtractionInvalid(errors)

• Cap repairs at 1-2 attempts. If it fails twice, route to a human queue or a stronger model; more retries rarely help.
• Feed back specific errors, not “invalid, try again”.
• For prompt-only fallback providers, parse with a JSON parser, strip markdown fences, validate with the same Pydantic/Zod model, and repair with the validation error text.

5. Evaluate

Measure per field on 50+ labeled documents: exact-match rate for IDs and enums, tolerance match for numbers, null-correctness (predicted null when gold is null, and not otherwise). Track the semantic-validation failure rate in production. See llm-evals.

Gotchas

• First request with a new schema can be slower while the provider compiles the grammar; schemas are cached afterwards, so do not generate a new schema per request.
• Free-text fields inside a strict schema still hallucinate. Shape is guaranteed, content is not.
• required plus no nullable option forces invented values for missing data.
• Large enums (hundreds of labels) hurt accuracy; use a two-stage classify (group, then label) or retrieval over label descriptions.
• Long documents with many repeated items (line items, rows) can hit max_tokens; check the stop reason and chunk the input.
• Assistant prefill tricks (starting the reply with {) are rejected by current Claude models; use the output format instead.
• Parse tool inputs with a JSON parser; string escaping of serialized inputs can differ between model versions.

Output contract

**Mechanism:** <output format | strict tool | both | prompt-only + repair>
**Schema:** <file/model name>, <n> fields, nullable fields <list>, enums <list>
**Validation:** <semantic rules>, repair attempts <n>, fallback <queue/model>
**Failure handling:** refusal -> <...>; max_tokens -> <...>
**Accuracy:** <field-level table on n labeled docs>