Rate Limiting skill for Claude Code

A good rate limiter protects capacity and fairness without surprising legitimate clients: limits are derived from real capacity and usage, enforced atomically across instances, communicated through standard headers, and paired with clients that back off. A limiter that races under concurrency or resets on deploy is decoration.

Ground rules

• Decide what you are protecting (capacity, cost, abuse, fairness between tenants) before picking numbers. Different goals need different keys and layers.
• Derive limits from data: current p99 usage per key and backend capacity. Start generous, log would-be rejections in shadow mode, then enforce.
• Distributed limits must be atomic: a single Lua script or a single atomic command, never read-then-write from the application.
• Always return 429 Too Many Requests with Retry-After when rejecting.
• Decide explicitly whether to fail open or closed when the limiter store is down.

Algorithm choice

Algorithm	Behavior	Memory per key	Use for
Token bucket	Allows bursts up to capacity, refills at a steady rate	O(1)	Default for public APIs; per-user/key limits
Sliding window counter	Weighted blend of current and previous window; smooth, approximate	O(1)	Per-minute/hour limits without boundary bursts
Fixed window counter	INCR + EXPIRE per window	O(1)	Coarse quotas (daily caps); accepts up to 2x burst at boundaries
Sliding log	Sorted set of request timestamps; exact	O(limit)	Low-volume, high-value limits (login attempts, OTP sends)
GCRA	Token-bucket equivalent storing one timestamp	O(1)	High-throughput limiters; same semantics as token bucket
Concurrency limit	Caps in-flight requests, not rate	O(1)	Expensive endpoints (exports, reports, LLM calls)
Leaky bucket (queue)	Smooths output to a fixed rate, adds latency	Queue	Shaping outbound calls to a strict provider

Workflow

- [ ] 1. State goals and pick keys and layers
- [ ] 2. Measure current usage per key; set limits with headroom
- [ ] 3. Choose algorithm per limit
- [ ] 4. Implement atomically (gateway config or Redis Lua)
- [ ] 5. Return 429 + Retry-After + rate limit headers
- [ ] 6. Ship in shadow mode, review would-be rejections, then enforce
- [ ] 7. Monitor rejections per key and route; alert on spikes

1. Keys and layers

Layer	Typical key	Purpose	Tool
Edge / CDN / WAF	IP, ASN, fingerprint	Abuse, scraping, DDoS-ish bursts	Cloudflare/CloudFront WAF rules, nginx limit_req
API gateway	API key, client ID	Plan limits, partner fairness	Gateway rate-limit plugins (Kong, Envoy, cloud API gateways)
Application	User, tenant, route, action	Business rules (5 password attempts, 100 exports/day)	Redis-backed limiter in code
Outbound	Provider + credential	Stay under third-party limits	Client-side token bucket + backoff

Layer them: coarse IP limits at the edge, precise per-key limits in the app. IP-only limits punish users behind shared NAT and are bypassed by rotating IPs; combine with authenticated keys wherever possible.

2. Setting numbers

• Pull per-key request rates for the last 2–4 weeks; set the limit above the p99 key’s peak (often 2–5x) so normal heavy users never hit it.
• Bucket capacity (burst) covers legitimate bursts such as page loads that fire 20 requests at once.
• Expensive routes get their own lower limits, not the global one.
• Per-plan limits live in config, not code, so sales can change them.

4. Implementation

nginx (per IP, edge-style):

limit_req_zone $binary_remote_addr zone=api:10m rate=10r/s;
server {
  location /api/ {
    limit_req zone=api burst=20 nodelay;
    limit_req_status 429;
  }
}

Application with Redis: use an atomic Lua script. Token bucket and sliding window counter scripts, with notes on Redis Cluster key slots and server time, are in references/redis-limiters.md; read it before writing a Redis limiter.

// usage sketch with ioredis
redis.defineCommand('tokenBucket', { numberOfKeys: 1, lua: TOKEN_BUCKET_LUA });

export async function limit(key: string, ratePerSec: number, capacity: number) {
  const [allowed, remaining, retryAfterMs] = await (redis as any).tokenBucket(
    `rl:{${key}}`, ratePerSec, capacity, 1);
  return { allowed: allowed === 1, remaining: Math.floor(Number(remaining)), retryAfterMs: Number(retryAfterMs) };
}

Libraries exist for most frameworks (e.g., express-rate-limit with a Redis store, django-ratelimit, Spring/Bucket4j, ASP.NET Core rate limiting middleware). Use one if it supports an atomic shared store and your algorithm; check how it computes keys behind proxies (X-Forwarded-For trust settings).

5. Response contract

HTTP/1.1 429 Too Many Requests
Retry-After: 12
RateLimit-Policy: "default";q=100;w=60
RateLimit: "default";r=0;t=12
Content-Type: application/problem+json

{"type":"https://api.example.com/problems/rate-limited","title":"Too many requests","status":429,"detail":"Limit of 100 requests per 60 s exceeded; retry in 12 s."}

• Retry-After (seconds or HTTP date) is the standard clients and SDKs honor.
• The RateLimit / RateLimit-Policy fields come from an IETF draft whose syntax has changed across revisions; many APIs still send X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset. Pick one convention, document it, and send it on successful responses too so clients can pace themselves.
• Use a distinct status for quota exhaustion vs. rate (both 429, different problem type), so clients know whether waiting seconds or upgrading a plan helps.

Outbound: respecting provider limits

• Read the provider’s documented limits and headers; pace with a client-side token bucket per credential so you rarely receive 429s.
• On 429/503: honor Retry-After; otherwise exponential backoff with full jitter and a cap (see idempotency-retries).
• Queue bulk work and drain at the allowed rate instead of bursting.

Fail open or closed?

Limit protects	Store unavailable →
Capacity or fairness for normal API traffic	Fail open (allow), alert loudly, keep a local in-memory fallback limit
Security (login attempts, OTP, password reset)	Fail closed or fall back to a strict local limit
Cost (paid LLM calls, SMS)	Fail closed

Set a short timeout (e.g., 20–50 ms) on limiter calls so a slow Redis does not become the outage.

Gotchas

• Read-then-write (GET count, compare, INCR) races under concurrency and lets bursts through. Use one atomic script.
• Using application server clocks across instances causes skew; use Redis TIME inside the script or a single time source.
• Fixed windows allow double the limit across a boundary (end of one minute + start of next).
• Trusting X-Forwarded-For from anywhere lets clients pick their own IP key; trust only your proxy hops.
• Lua returns numbers to Redis as integers (truncated); return fractional values as strings.
• Limits per instance (in-memory) multiply with the instance count after autoscaling.
• Retries from your own clients count against limits; retry storms can lock out a whole tenant.

Verification

1. Unit-test the algorithm with a controllable clock: exact boundary, burst, refill.
2. Concurrency test: fire 2x the limit in parallel from several processes; accepted count equals the limit (± algorithm tolerance).
3. Responses include Retry-After and the chosen rate limit headers; clients back off.
4. Kill the limiter store in staging and confirm the chosen fail-open/closed behavior.