Blue-Green and Canary Deploys skill for Claude Code

A release strategy is done when a new version reaches users gradually or switchably, promotion is gated on metrics comparing new vs. old, rollback is a single tested action that takes under five minutes, and the database works with both versions at once.

Ground rules

• Both versions must run side by side safely: schema changes are expand/contract, message formats are backward and forward compatible, caches tolerate both.
• Promotion depends on metrics for the new version specifically (error rate, latency, saturation, a business metric), not on “no one complained”.
• Write the rollback plan before deploying, and rehearse it in staging.
• Shifting production traffic, promoting, or aborting requires explicit user confirmation unless an automated analysis gate makes the decision by design.
• Keep the old version running until the new one has baked; tear down only after.

Choosing a strategy

Strategy	Use when	Cost / risk
Rolling update	Stateless service, low risk change, good readiness probes	Cheapest; rollback is another rollout; mixed versions during rollout
Blue/green	Need instant full switch and instant rollback; can afford 2x capacity briefly	Double capacity; all users hit new version at once
Canary	High-traffic service, want real-user evidence before full exposure	Needs per-version metrics and traffic splitting
Feature flag / dark launch	Risk is in a feature, not the binary; want per-user/tenant targeting	Flag debt; code paths for both states
Shadow / mirroring	Validate a rewrite against real traffic without user impact	Side effects must be suppressed; doubles load

Default: canary with automated analysis for customer-facing services with enough traffic (roughly 100+ requests/min to the canary at its first step); blue/green for low-traffic services where canary statistics would be meaningless; flags on top of either for risky features.

Workflow

- [ ] 1. Check compatibility: DB migrations, APIs, messages, caches, sessions
- [ ] 2. Pick strategy and traffic-splitting mechanism
- [ ] 3. Define promotion metrics and thresholds
- [ ] 4. Write the rollback plan (trigger, command, owner, data caveats)
- [ ] 5. Deploy new version with zero or minimal traffic; smoke test it directly
- [ ] 6. Shift traffic in steps with bake time and analysis
- [ ] 7. Promote, keep old version warm, then clean up

1. Compatibility checklist

• Migrations are additive (new nullable columns, new tables); destructive steps deferred to a later release (see schema-migrations).
• Old code ignores new columns/fields; new code handles missing ones.
• Queue consumers of both versions can read messages produced by either.
• Session and auth tokens valid across versions; no sticky-session dependency.
• Scheduled jobs/cron will not run twice (both colors) — disable on the idle color.
• Cache keys versioned if value shapes changed.

3. Promotion metrics

Metric	Gate example
Error rate (5xx / total) for new version	< 1% and not > 1.5x the stable version
Latency p95/p99 for new version	Within 10–20% of stable
Saturation (CPU, memory, restarts)	No restarts; memory not trending up
Business metric (checkout success, sign-ups)	No significant drop vs. stable

Compare canary vs. stable over the same window, not canary vs. yesterday.

Kubernetes: Argo Rollouts canary

apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata: { name: api }
spec:
  replicas: 10
  selector: { matchLabels: { app: api } }
  template:          # same pod template as a Deployment
    metadata: { labels: { app: api } }
    spec:
      containers:
        - name: api
          image: registry.example.com/api:1.15.0
  strategy:
    canary:
      stableService: api-stable
      canaryService: api-canary
      trafficRouting:
        nginx: { stableIngress: api }
      steps:
        - setWeight: 5
        - pause: { duration: 10m }
        - analysis: { templates: [{ templateName: error-rate }], args: [{ name: svc, value: api-canary }] }
        - setWeight: 25
        - pause: { duration: 10m }
        - setWeight: 50
        - pause: { duration: 15m }
---
apiVersion: argoproj.io/v1alpha1
kind: AnalysisTemplate
metadata: { name: error-rate }
spec:
  args: [{ name: svc }]
  metrics:
    - name: error-rate
      interval: 1m
      count: 5
      failureLimit: 1
      successCondition: result[0] < 0.01
      provider:
        prometheus:
          address: http://prometheus.monitoring:9090
          query: |
            sum(rate(http_requests_total{service="{{args.svc}}",code=~"5.."}[2m]))
            / sum(rate(http_requests_total{service="{{args.svc}}"}[2m]))

kubectl argo rollouts get rollout api -n shop --watch
kubectl argo rollouts promote api -n shop        # skip current pause (confirm first)
kubectl argo rollouts abort api -n shop          # shift all traffic back to stable

Flagger offers the same model driven by a Canary resource. Gateway API can split traffic natively with weighted backendRefs on an HTTPRoute.

Kubernetes: plain blue/green

Two Deployments (api-blue, api-green) labeled version: blue|green; the Service selects one:

kubectl rollout status deploy/api-green -n shop
kubectl port-forward deploy/api-green 8080:8080 -n shop   # smoke test green directly
kubectl patch service api -n shop -p '{"spec":{"selector":{"app":"api","version":"green"}}}'   # switch (confirm)
# rollback: patch the selector back to blue

AWS

• ALB weighted target groups (gradual or instant):

aws elbv2 modify-listener --listener-arn "$LISTENER" --default-actions \
  '[{"Type":"forward","ForwardConfig":{"TargetGroups":[
     {"TargetGroupArn":"'"$TG_BLUE"'","Weight":90},
     {"TargetGroupArn":"'"$TG_GREEN"'","Weight":10}]}}]'

• ECS: CodeDeploy blue/green or ECS native blue/green deployments with test listeners and lifecycle hooks; set CloudWatch alarms as automatic rollback triggers.
• Lambda: alias traffic shifting with CodeDeploy configs such as LambdaCanary10Percent5Minutes, with alarms for automatic rollback.

Feature-flagged rollout

• Ship code dark (flag off), enable for internal users, then 1% -> 10% -> 50% -> 100% of users by a stable hash of user or tenant ID (consistent experience per user).
• Use an OpenFeature-compatible SDK or the existing flag vendor; evaluate server-side for anything security- or billing-relevant.
• Kill switch documented in the runbook; flag removal ticket created at 100%.

4. Rollback plan template

## Rollback plan — api 1.15.0
- Trigger: canary error rate > 1% or p99 > 1.2x stable for 5 min, or any data integrity alert
- Action: `kubectl argo rollouts abort api -n shop` (automatic on failed analysis)
- Time to effect: < 1 min (traffic weight change); pods stay warm
- Data caveats: 1.15.0 writes new column `orders.channel`; 1.14.x ignores it (safe)
- Not reversible by rollback: emails sent, external webhooks fired
- Owner: on-call for shop; approver: release manager

Verification

• Smoke-test the new version directly (port-forward, test listener, preview URL) before it gets real traffic.
• At each step, read the per-version dashboard: error rate, latency, saturation.
• After promotion, confirm 100% of traffic reaches the new version (sum by (version) (rate(http_requests_total[5m]))).
• Rehearse the rollback in staging and record how long it took.

Gotchas

• Rollback does not undo migrations or data written by the new version.
• Long-lived connections (WebSockets, gRPC streams) stay on the old color until they reconnect; plan a drain or forced reconnect.
• Sticky sessions skew canary percentages; low traffic makes canary stats noise.
• DNS-based blue/green is slow and uneven due to TTLs and client caching; switch at the load balancer instead.
• Both colors consuming the same queue process messages with mixed logic; pause or version consumers.
• Warm-up: a cold green environment can fail on the first full load (JIT, caches, connection pools). Pre-warm or shift gradually.

Output

**Release:** api 1.14.3 -> 1.15.0, canary via Argo Rollouts (5/25/50/100, 10–15 min bakes)
**Gates:** error rate < 1% (Prometheus analysis), p99 within 20% of stable
**Compatibility:** migration 0042 additive; consumers tolerant; cron disabled on canary
**Result:** promoted at 15:40 UTC; canary error rate 0.21% vs stable 0.19%
**Rollback:** `kubectl argo rollouts abort api -n shop` (rehearsed in staging: 40 s)