Prompt Caching and Costs skill for Claude Code

Cost work is done when you have a measured token profile (where spend goes by feature, input vs. output, cached vs. uncached), the free wins are applied (caching, batching, trimming), any quality-trading change (cheaper model, lower effort) passed the eval, and the savings show up in logged usage, not in a spreadsheet estimate.

Ground rules

• Measure before optimizing. Pull usage from logs or the provider’s usage reports and rank features by spend.
• Free wins first, trade-offs second: caching, batching, and trimming do not change output quality; model downgrades and effort cuts might.
• Judge cost per completed task, not per request. A cheaper call that needs retries or extra agent turns is not cheaper.
• Every experiment that calls the model costs money. Estimate it and get the user’s approval before large eval runs.

Workflow

- [ ] 1. Build the token profile
- [ ] 2. Fix prompt caching (structure, breakpoints, invalidators)
- [ ] 3. Move non-urgent work to the batch API
- [ ] 4. Trim input: history, retrieved context, tool results, tool definitions
- [ ] 5. Control output: max_tokens, verbosity, reasoning effort
- [ ] 6. Route by difficulty (only with eval evidence)
- [ ] 7. Verify savings in production usage

1. Token profile

For each feature, collect over a representative window: request count, mean and p95 input tokens, cache_read_input_tokens, cache_creation_input_tokens, output tokens, model. Then model the options:

python3 scripts/cost_model.py --model claude-sonnet-5-5 --requests 200000 \
  --prefix-tokens 6000 --variable-tokens 800 --output-tokens 400 --hit-rate 0.9 --batch-share 0.3

The script prints spend with and without caching and batching, the output share of spend, the break-even cache hit rate for the chosen TTL, and a what-if across model tiers. Its built-in prices are a snapshot; confirm them on the provider’s pricing page or pass --input-price/--output-price/--cache-read-mult.

Read the profile:

Finding	First lever
Large repeated prefix (system prompt, tools, documents), low cache reads	Caching (step 2)
Offline/background volume	Batch API (step 3)
Input grows with conversation length or agent turns	Trimming (step 4)
Output > 50% of spend	Output control and effort (step 5)
Simple, high-volume requests on a top-tier model	Routing (step 6)

2. Prompt caching

Caching is a prefix match: the provider reuses computation for an identical leading sequence of tokens. Render order on the Anthropic Messages API is tools -> system -> messages. Any byte change in the prefix invalidates everything after it.

Structure every request as stable first, volatile last:

resp = client.messages.create(
    model="claude-sonnet-5-5",
    max_tokens=1024,
    tools=TOOLS,                                   # fixed list, fixed order
    system=[
        {"type": "text", "text": STABLE_INSTRUCTIONS},
        {"type": "text", "text": REFERENCE_DOCS,
         "cache_control": {"type": "ephemeral"}},  # breakpoint: caches tools + system
    ],
    messages=history + [{"role": "user", "content": user_msg}],
)
u = resp.usage
log.info("cache", read=u.cache_read_input_tokens, write=u.cache_creation_input_tokens, uncached=u.input_tokens)

Placement rules (Anthropic specifics; other providers cache prefixes automatically or with their own controls, but the same ordering principle applies):

• Up to 4 cache_control breakpoints per request. Typical: end of system/tools, and the last message of the conversation so each turn reads the previous turn’s prefix.
• A top-level cache_control on the request auto-places one breakpoint on the last cacheable block; use it when you do not need fine-grained placement.
• Prefixes below the model’s minimum length (from 512 tokens on the newest Claude models up to 4,096 on some older ones) silently do not cache.
• Default TTL is 5 minutes, refreshed on every read. {"type": "ephemeral", "ttl": "1h"} costs more to write (2x vs. 1.25x base input) and only pays off when requests sharing the prefix arrive 5-60 minutes apart.
• Cache reads are billed at a small fraction of base input price (about 0.1x, lower on some current models) and writes at a premium, so a prefix pays off after roughly two requests at the 5-minute TTL.
• Caches are scoped per model and per workspace/organization. Splitting traffic across models or workspaces splits the cache.

Silent invalidators to grep for:

Invalidator	Fix
Timestamp, date, request ID, or user name in the system prompt	Move to the final user message
Non-deterministic JSON (unsorted keys) in the prefix	json.dumps(obj, sort_keys=True)
Tool list built from a set or filtered per request	Fixed, sorted tool list per session
Retrieved documents reordered between turns	Append new context; do not reshuffle earlier blocks
Editing earlier messages (summaries rewritten each turn)	Append-only history; summarize in larger, rarer steps
Switching model, reasoning effort, or tool_choice mid-session	Keep per-session settings fixed

Verify: send the same request twice within the TTL. The second response must show cache_read_input_tokens > 0. If it is zero, diff the two rendered request bodies byte for byte.

3. Batch API

Anthropic’s Message Batches API processes requests asynchronously (most finish within an hour, up to 24 hours) at about half the price, and stacks with caching. Use it for evals, backfills, nightly summarization, classification of stored data, and embedding-adjacent enrichment. Key results by custom_id; they return in any order.

batch = client.messages.batches.create(requests=[
    {"custom_id": f"doc-{d.id}", "params": {"model": "claude-haiku-5-5", "max_tokens": 512,
     "system": SYSTEM, "messages": [{"role": "user", "content": d.text}]}}
    for d in docs
])
# poll client.messages.batches.retrieve(batch.id).processing_status until "ended",
# then iterate client.messages.batches.results(batch.id)

4. Trim input

• Chat history: keep the last N turns verbatim plus a summary of older turns; or use the provider’s server-side compaction/context-editing features for long agent sessions.
• RAG: send the top 5-8 reranked chunks, not 30.
• Agent tool results: cap result size; clear or summarize old tool results.
• Tool definitions count as input on every call. Remove unused tools; load domain subsets.
• Count tokens with the provider’s counting endpoint (client.messages.count_tokens); tokenizers differ across vendors and model generations.

5. Control output

• Output tokens cost several times more than input. Set max_tokens to the real need and ask for concise formats (“3 bullets”, “label only”).
• Reasoning tokens bill as output. On current Claude models tune output_config: {"effort": ...} per route: low for classification and chat, higher for coding and agentic work. Measure; do not guess.
• For classification, return a label from a structured output, not an explanation.

6. Route by difficulty

• Route cheap/easy traffic to a smaller model (claude-haiku-5-5) and hard traffic to a larger one, via a rule (feature, input length) or a cheap classifier.
• First test the simpler alternative: the stronger model at lower effort may match a cascade with one cache namespace and less code.
• Every route needs its own eval pass on real traffic samples (see llm-evals).

7. Verify in production

Compare a week of usage before and after: spend per feature, cache hit ratio (cache_read / (cache_read + cache_creation + input) on cacheable traffic), output tokens per request, and cost per completed task. Confirm the eval score held.

Gotchas

• A 1-hour TTL on continuous traffic just doubles the write cost; reads already keep the 5-minute entry warm.
• Caching a prefix that changes every request (hit rate below break-even) costs more than not caching. The script’s break-even line flags this.
• Generation time counts against the TTL; a 4-minute response leaves 1 minute for the next request to hit a 5-minute cache.
• Parallel first requests all miss: warm the cache with one request before fanning out.
• Downgrading a model to save money without an eval is a quality regression waiting for a customer to report it.

Output contract

**Profile:** <period> spend <$>; top features <name: $ and %>; output share <x%>
**Applied (free):** caching (hit ratio <a> -> <b>), batch share <c>, trimming <what>
**Applied (trade-offs):** <model/effort changes> — eval <score before -> after>
**Measured savings:** <$ per period, %> from production usage, cost per task <x -> y>