LLM Batch Jobs skill for Claude Code

A batch job is done when the reconciliation report shows every input ID with exactly one final state (succeeded and written, permanently failed with a reason, or deliberately skipped), the output passed validation, and actual cost matched the estimate within a few percent. “The script finished” is not done; batch results arrive out of order, some requests error or expire, and reruns double-write unless the design prevents it.

Ground rules

• Pilot on 50-200 items synchronously and check quality before submitting 100,000.
• Every request gets a deterministic ID derived from the item and the job version, so reruns and retries are idempotent.
• Results are matched by ID, never by position.
• Writes are upserts keyed by item ID plus job version. A rerun must not create duplicates or overwrite newer results with older ones.
• Keep a manifest of what was submitted and a state store of what came back. Never rely on memory or terminal scrollback to know what is left.
• Estimate cost before submitting and get the user’s go-ahead for large spends.

Batch API or concurrent workers?

Need	Use
Results within minutes, interactive	Synchronous calls with bounded concurrency
Thousands+ items, results within hours are fine	Batch API (Anthropic Message Batches: roughly half price, most batches finish within an hour, hard limit 24 h)
Item N depends on item N-1’s output	Sequential pipeline; batch per stage
Very large job with a deadline	Batch API, chunked; monitor and resubmit stragglers

Anthropic Message Batches facts to plan around (check the docs for current limits): up to 100,000 requests or 256 MB per batch, results downloadable for 29 days, results in any order, every Messages API feature allowed inside a request (tools, images, PDFs, structured outputs, prompt caching). Some request parameters, such as server-side fallbacks, are not accepted in batches.

Workflow

- [ ] 1. Freeze the prompt, model, and output schema; pilot synchronously
- [ ] 2. Build the manifest with deterministic custom IDs
- [ ] 3. Estimate cost and get approval
- [ ] 4. Submit in chunks; record batch IDs
- [ ] 5. Poll; download results when ended
- [ ] 6. Validate outputs; classify failures
- [ ] 7. Retry what is retryable (new batch, same IDs)
- [ ] 8. Write idempotently; reconcile; report

1. Pilot

Run 50-200 representative items through the exact request you will batch. Grade them (see llm-evals). Fix the prompt now; changing it after submitting means a new job version and paying again.

2. Manifest

import hashlib, json

JOB = "ticket-labels-v3"          # bump when prompt, model, or schema changes

def custom_id(item_id: str) -> str:
    # letters, digits, hyphen, underscore; keep it short (the API caps the length)
    return hashlib.sha256(f"{JOB}:{item_id}".encode()).hexdigest()[:40]

with open("manifest.jsonl", "w") as out:
    for item in items:
        out.write(json.dumps({"custom_id": custom_id(item["id"]), "item_id": item["id"],
                              "job": JOB}) + "\n")

Store custom_id -> item_id because the API only echoes custom_id.

3. Estimate cost

• Count tokens for 100 sampled requests with the token counting endpoint; extrapolate input tokens. Estimate output from the pilot’s actual usage.output_tokens.
• Apply the batch discount and expected cache hits (shared system prompt placed first with cache_control; cache hits inside batches are best-effort, so budget conservatively).
• Show the user: items, estimated input and output tokens, estimated cost, and model. Large spends need an explicit yes.

4. Submit (Anthropic Python SDK)

import anthropic
from anthropic.types.message_create_params import MessageCreateParamsNonStreaming
from anthropic.types.messages.batch_create_params import Request

client = anthropic.Anthropic()
SYSTEM = [{"type": "text", "text": INSTRUCTIONS, "cache_control": {"type": "ephemeral"}}]

def to_request(item):
    return Request(
        custom_id=custom_id(item["id"]),
        params=MessageCreateParamsNonStreaming(
            model="claude-haiku-5-5",
            max_tokens=512,
            system=SYSTEM,
            messages=[{"role": "user", "content": f"<ticket>{item['text']}</ticket>"}],
            output_config={"format": {"type": "json_schema", "schema": SCHEMA}},
        ),
    )

def submit(chunk):
    batch = client.messages.batches.create(requests=[to_request(i) for i in chunk])
    state.record_batch(JOB, batch.id, [custom_id(i["id"]) for i in chunk])
    return batch.id

• Chunk at 10,000-50,000 requests per batch rather than the maximum: failures are smaller, results arrive sooner, and you stay well under the size cap with large inputs.
• Record the batch ID and the custom IDs it contains before doing anything else.
• Skip items whose ID already has a succeeded result for this job version.

5. Poll and download

import time
def wait(batch_id, every=60):
    while True:
        b = client.messages.batches.retrieve(batch_id)
        if b.processing_status == "ended":
            return b
        time.sleep(every)

def download(batch_id, path):
    with open(path, "w") as out:
        for r in client.messages.batches.results(batch_id):
            out.write(r.model_dump_json() + "\n")       # keep raw results on disk

Save raw results before processing them, so a parsing bug never forces a re-run.

6. Validate and classify

Result	Meaning	Action
succeeded, output valid	Done	Write
succeeded, stop_reason = max_tokens	Truncated	Retry with higher max_tokens
succeeded, stop_reason = refusal	Model declined	Record reason; human review, do not loop
succeeded, output fails schema or business rules	Bad output	Retry once; then mark failed
errored, invalid request	Your request is malformed	Fix the request builder; do not blind-retry
errored, server/overloaded	Transient	Retry in a new batch
expired	Not processed within 24 h	Resubmit
canceled	You canceled	Resubmit if still wanted

7. Retry

Build a retry manifest from failures (the bundled script does this), submit as a new batch with the same custom IDs, and cap retries per item (2-3). Items that fail the cap go to a dead-letter file with their last error.

8. Write, reconcile, report

python3 scripts/reconcile.py manifest.jsonl results/*.jsonl \
  --require-json --retry-out retry.jsonl

The script joins the manifest against all downloaded result files (later files win for the same custom_id), counts each final state, lists missing IDs and unknown IDs (results not in the manifest), flags succeeded results whose text is not valid JSON (--require-json), sums token usage, and writes a retry manifest for errored, expired, truncated, invalid, and missing items. It exits 1 if anything is unaccounted for.

Write results with an upsert keyed by (item_id, job), storing model, prompt version, and batch ID alongside the output for lineage.

Concurrency workers (when not using batch)

• Bounded concurrency (semaphore) sized to your rate limits; start low and raise.
• Respect retry-after on 429; exponential backoff with jitter on 5xx and overloaded errors; the SDKs retry a few times by default.
• Persist progress per item so a crash resumes instead of restarting.
• Same deterministic IDs, validation, and reconciliation as the batch path.

Gotchas

• Changing the prompt mid-job silently mixes two versions in one dataset. Bump the job version and keep both labeled.
• Results files and batch metadata expire; download promptly and keep raw copies.
• Huge inline inputs (PDFs, images) hit the batch size cap fast; upload once with a files API where available and reference by ID.
• A request that is malformed fails in every retry. Separate invalid-request errors from transient ones before retrying.
• Rate limits for batch queues are separate from synchronous limits; very large jobs may queue. Chunk and stagger submissions.
• Writing results as they stream in without the custom-ID map creates orphans you cannot attribute.

Output contract

**Job:** <name/version> | model <id> | prompt <hash>
**Items:** <n> in manifest | batches <ids>
**Final states:** succeeded+written <n>, failed permanent <n>, refused <n>, skipped <n>, unaccounted 0
**Retries:** <n> items retried, <n> dead-lettered (file <path>)
**Tokens:** input <n> (cached <n>), output <n> | **Cost:** est <$> vs actual <$>
**Quality:** pilot grade <x>; post-run sample <y> of <n> checked