28 skills · 3 categories
Claude Code skills for LLM and AI apps
Skills for building with language models: API integration with streaming and retries, evaluation sets, retrieval pipelines, prompt-injection defense, agents and cost control. Claude Code skills about agents and subagents appear here as well.
Start with these
- 1
LLM App Builder
Ship LLM features with timeouts, streams, and budgets
Claude builds LLM features the way a production team would: one client module with timeouts and typed retries, streaming for anything a user waits on, a model picked by a small eval instead of reputation, and per-request cost logged from the first deploy. Every stop reason, including truncation and refusal, gets explicit handling and a test.
- 2
LLM Batch Jobs
Every input accounted for, at batch prices
Claude runs bulk LLM work so nothing is lost or double-written. It pilots the prompt first, builds a manifest with deterministic custom IDs, estimates cost before submitting, chunks work into batches, saves raw results, and sorts failures into retryable and needs-a-human. A bundled script reconciles the manifest against every results file and writes the retry list automatically.
- 3
LLM Evals
Know whether the change helped, with error bars
Claude builds evals that can actually decide things. Cases come from real traffic and past failures, each criterion gets the cheapest reliable grader, and any LLM judge is checked against human labels before its scores count. Version comparisons are paired on the same cases with a McNemar test and bootstrap interval, and a bundled script fails CI when must-pass cases regress.
All 28 LLM and AI apps skills
Grouped by category. Each card opens the full playbook, example prompts and install commands.
Claude Code Mastery 2
Agent Eval Harness
Prove a skill or prompt change helped, run after run
Instead of eyeballing one run, Claude writes a small labeled suite of positive, negative and edge cases from real usage, grades each with deterministic checks on outcome and process before reaching for a judge model, runs every case several times through claude -p or claude plugin eval, and compares per case against a committed baseline. A stdlib runner script handles fixtures, tool-call grading and regression exit codes.
Skill Description Tuning
Make skills fire on the right prompts, and only those
Treats a skill description as a routing classifier and tests it like one. Claude builds a labeled set of realistic prompts and near-miss negatives, runs each in a fresh headless session to see whether the skill actually fires, diagnoses the miss pattern, and rewrites one variable at a time until recall is high, false positives are zero, and sibling skills still win their own requests.
AI Engineering 25
LLM App Builder
Ship LLM features with timeouts, streams, and budgets
Claude builds LLM features the way a production team would: one client module with timeouts and typed retries, streaming for anything a user waits on, a model picked by a small eval instead of reputation, and per-request cost logged from the first deploy. Every stop reason, including truncation and refusal, gets explicit handling and a test.
LLM Batch Jobs
Every input accounted for, at batch prices
Claude runs bulk LLM work so nothing is lost or double-written. It pilots the prompt first, builds a manifest with deterministic custom IDs, estimates cost before submitting, chunks work into batches, saves raw results, and sorts failures into retryable and needs-a-human. A bundled script reconciles the manifest against every results file and writes the retry list automatically.
LLM Evals
Know whether the change helped, with error bars
Claude builds evals that can actually decide things. Cases come from real traffic and past failures, each criterion gets the cheapest reliable grader, and any LLM judge is checked against human labels before its scores count. Version comparisons are paired on the same cases with a McNemar test and bootstrap interval, and a bundled script fails CI when must-pass cases regress.
LLM Classification
Labels that match careful humans, per class
Claude builds classifiers starting from the taxonomy, not the prompt. It writes label definitions with boundary examples and an escape label, measures human agreement with Cohen's kappa, constrains outputs to an enum, and reports per-class precision and recall on a frozen test set. Uncertain items escalate through a model cascade or to humans, and a bundled script computes kappa, confusion matrices, and macro-F1.
LLM Gateway Routing
Fallbacks that are tested, retries that cannot snowball
Claude treats multi-provider routing as a contract per route rather than a list of model names. It builds an error-to-action table, splits time-to-first-token from total deadlines, caps retries with a budget, checks that each fallback passes the route's eval, and picks between an in-app router, a LiteLLM proxy, or a managed gateway, including when you need none of them.
LLM Observability
Trace any bad answer back to its exact prompt
Claude wraps every model, retrieval and tool call in traced, versioned spans with tokens, cost, cache reads, latency and stop reason, so any reported bad answer leads to its exact rendered prompt. Content logging follows an explicit redaction, sampling and retention policy, dashboards and alerts cover spend and truncation, and trace-linked user feedback becomes new eval cases.
Document Extraction with LLMs
Invoices to validated JSON, with evidence per field
Claude turns invoices, receipts, and contracts into structured records you can trust. It designs nullable schemas with verbatim evidence for key fields, extracts with schema-constrained output from native PDFs or scans, checks totals, dates, and IDs in code, and routes anything doubtful to human review. A bundled scorer reports per-field accuracy, misses, and hallucinations against a gold set.
Prompt Engineering
Prompts measured against evals, not eyeballed
Claude stops tweaking prompts by feel. It gathers real inputs and gradeable criteria first, drafts with a proven skeleton of tagged context, instructions, constraints and examples, then fixes failures one class at a time while re-running a held-out eval. Prompt revisions ship with a version number, a changelog line per edit and before and after scores.
RAG Pipelines
Fix retrieval first, then make answers cite it
Claude treats RAG as two systems with two scorecards. It builds a labeled question set, chunks with document structure and context prefixes, retrieves with hybrid search fused by reciprocal rank and a reranker, and generates answers that cite supplied sources or abstain. A bundled script reports recall@k, MRR and nDCG so retrieval failures are fixed before anyone touches the prompt.
Fine-Tuning
Fine-tune only after the prompt baseline plateaus
Claude starts by asking whether fine-tuning is the right tool, measuring the best prompt and retrieval setup first. When tuning is warranted it builds deduplicated, consistently labeled chat datasets with leak-free splits, trains LoRA or QLoRA with TRL using conservative defaults and val-loss checkpointing, and compares the result to the prompted baseline through the real serving stack.
Vector Search
ANN tuned against exact search, with filters on
Claude picks a vector store by operational fit, creates indexes with explicit HNSW or IVFFlat parameters and filter indexes, and then proves the setup by comparing ANN results to brute-force search on real queries with production filters. It sweeps ef_search to the recall and latency knee and flags post-filtering, mixed embedding models and unnormalized vectors before they ship.
Local LLMs
Size it, quantize it, benchmark it, then serve
Claude sizes memory before anything downloads, using a bundled estimator that reads the model's config to compute weights plus KV cache for your context and concurrency. It then picks a runtime and quantization, sets context length and GPU offload explicitly, benchmarks prefill and decode speed, checks quality on your own tasks and keeps the endpoint on localhost or behind auth.
Prompt Caching and Costs
Cut the LLM bill with caching before downgrades
Claude profiles where tokens actually go before changing anything, then applies the no-quality-loss levers in order: stable-prefix prompt caching with correct breakpoints, the batch API for offline work, and trimmed history and tool results. A bundled cost model shows break-even hit rates and tier what-ifs, and cheaper models or lower effort ship only after passing the eval.
Agent Memory
Remember what matters, forget on request
Claude designs agent memory from recall tests outward: which layers you actually need, a schema with provenance and expiry, a write policy that refuses secrets and untrusted content, conflict handling when facts change, and deletion that purges vectors and summaries too. It red-teams memory poisoning and cross-user leakage before anything ships.
Multi-Agent Orchestration
Parallel agents only where they pay for themselves
Claude first checks whether multiple agents are justified at all, naming the bottleneck they remove. When they are, it writes self-contained worker briefs with scope boundaries and a result schema, fans out under a concurrency cap with cheap worker models, and synthesizes with provenance and explicit gaps. The design is kept only if it beats a single-agent baseline on quality or wall-clock.
Prompt Injection Defense
Contain what an injected instruction can do
Claude defends LLM apps on the assumption that some injection will get through. It maps every untrusted input, breaks the private data, untrusted content and egress trifecta per agent, enforces permissions and approvals in code, closes markdown-image and URL exfiltration channels, and builds an adversarial suite from a catalog of real attack categories that runs in CI.
Recommendation Systems
Beat the popularity baseline where it counts: online
Claude builds recommenders in the order that pays off: impression logging first, popularity and co-occurrence baselines next, then candidate generators measured by recall and a learning-to-rank model with point-in-time features. It evaluates with time-based splits, watches coverage and novelty so results do not collapse onto bestsellers, handles cold start, and trusts the A/B test over offline gains.
Synthetic Data
Generated data that beats real-only on real tests
Claude generates synthetic datasets that have to prove themselves. It plans diversity with attribute grids and rotated real seeds, validates every row with code, scores quality with a judge calibrated against humans, and strips exact and near-duplicates plus anything overlapping your eval set using a bundled MinHash script. The data is kept only if it improves results on real held-out data.
Tool-Use Agents
Tools the model can pick right, loops that stop
Claude designs agent tools the way the model needs them: few high-level tools with descriptions that say when to use them, strict schemas, size-capped results, and errors phrased as next steps. The agent loop returns all parallel results in one message, enforces turn, token and time budgets, detects repeated calls, and gates refunds, emails and deletes behind explicit approval.
Model Serving
Self-hosted LLM endpoints sized to a real SLO
Claude deploys open-weight models like a platform engineer. It sizes GPU memory from the model config, weights, and KV cache, picks parallelism and quantization only after checking quality on your eval, tunes batching knobs with a load test shaped like real traffic, and autoscales replicas on queue depth with measured cold-start time. You end with cost per million tokens and an alert set.
Content Moderation
Policy first, thresholds from data, humans for the gray zone
Claude starts moderation work from a written policy with near-miss examples, not from a toxicity score. It builds a labeled evaluation set, layers normalization, blocklists, a moderation API, an LLM judge and a human queue, sets thresholds per category from precision and recall targets, and adds appeals, statements of reason and an audit log that records the policy version behind each decision.
Hugging Face Transformers
Pinned models, right templates, correct padding
Claude uses the Hugging Face stack without the classic silent bugs. It pins model revisions, loads safetensors in an explicit dtype and device map, formats prompts with the model's own chat template, pads left for batched generation and decodes only new tokens. Datasets are mapped in batches, classifiers trained with Trainer, and every pipeline is checked against known-good outputs.
Image Generation
Consistent image sets, priced per accepted image
Claude builds image pipelines that produce usable sets rather than lucky samples. It picks a hosted API or open diffusion model by task and license, writes structured prompt templates, locks consistency with seeds, references, or LoRAs, uses a vision LLM as a calibrated critic, moderates prompts and outputs, composites real logos and text afterward, and tracks cost per accepted image.
Realtime Voice Agents
Answer fast, stop when interrupted, never talk over callers
Claude builds voice agents around a measured voice-to-voice budget rather than vendor demo numbers. It picks a cascaded or speech-to-speech design, tunes VAD or semantic turn detection, cancels and truncates speech on barge-in so history matches what the caller heard, covers slow tools with spoken filler, and replays a recorded-audio suite in real time to catch regressions.
Structured Outputs
Schema-valid JSON, then checks on what it says
Claude replaces prompt-and-regex JSON with native constrained decoding: a JSON schema output format or strict tool generated from Pydantic or Zod models. It designs schemas with nullable fields and escape-value enums so the model never invents data, checks stop reasons before parsing, validates meaning as well as shape, and caps repair retries before routing to a human.
Install
Every LLM and AI apps skill in one command
Installs into ~/.claude/skills/. Add --project after -- to install into the current repository.
curl -fsSL https://claudeskills.page/install.sh | sh -s -- llm-app-builder llm-batch-jobs llm-evals llm-classification llm-gateway-routing llm-observability document-extraction-llm prompt-engineering rag-pipelines fine-tuning vector-search local-llms prompt-caching-costs agent-eval-harness ai-search-optimization agent-memory multi-agent-orchestration prompt-injection-defense recommendation-systems synthetic-data tool-use-agents skill-description-tuning model-serving content-moderation huggingface-transformers image-generation realtime-voice-agents structured-outputs LLM and AI apps skills FAQ
What is the best Claude Code skill for LLM and AI apps?
Start with LLM App Builder: Ship LLM features with timeouts, streams, and budgets. Then add LLM Batch Jobs and LLM Evals, which cover the next most common LLM and AI apps requests.
How do I install all 28 LLM and AI apps skills at once?
Pass every name to the installer in one command: curl -fsSL https://claudeskills.page/install.sh | sh -s -- llm-app-builder llm-batch-jobs llm-evals llm-classification llm-gateway-routing llm-observability document-extraction-llm prompt-engineering rag-pipelines fine-tuning vector-search local-llms prompt-caching-costs agent-eval-harness ai-search-optimization agent-memory multi-agent-orchestration prompt-injection-defense recommendation-systems synthetic-data tool-use-agents skill-description-tuning model-serving content-moderation huggingface-transformers image-generation realtime-voice-agents structured-outputs. Add --project to install them into the current repository's .claude/skills/ folder so your team gets them too.
Do these LLM and AI apps skills work outside Claude Code?
Yes. Each skill uses only the portable Agent Skills frontmatter, so the same folder works on claude.ai, through the Claude API and in the Agent SDK. Skills that run bundled scripts need an environment with code execution.