Event-Driven Architecture skill for Claude Code

Correct event-driven systems assume every message can arrive late, twice, or out of order, and still converge. Done well: state changes and their events commit atomically (outbox), consumers are idempotent (inbox), long workflows have explicit compensations, and schemas evolve compatibly.

Ground rules

• Never write to the database and publish to a broker as two independent steps. Use an outbox (or CDC) so both happen or neither does.
• Every consumer is idempotent. At-least-once delivery is the norm; exactly-once is a property you build, not one you are given.
• Events are facts in past tense, owned by the producer (OrderPlaced). Commands are requests to one handler (ReserveStock). Do not disguise commands as events.
• Ordering is only guaranteed within a partition/key. Design for that scope explicitly.
• Every event has an ID, type, version, source, occurred-at timestamp, and a correlation ID.

Workflow

- [ ] 1. List the business facts (events) and requests (commands)
- [ ] 2. Choose event style per flow: notification vs. state transfer
- [ ] 3. Define the envelope and schema with a version
- [ ] 4. Publish via transactional outbox (or CDC)
- [ ] 5. Make consumers idempotent; set retry and DLQ policy
- [ ] 6. Design sagas for multi-step workflows with compensations
- [ ] 7. Plan schema evolution and replay
- [ ] 8. Add observability: lag, DLQ depth, end-to-end latency

1–2. Messages and styles

Kind	Semantics	Example	Routing
Event	Fact, many subscribers, producer does not care who listens	PaymentCaptured	Topic / pub-sub
Command	Request, exactly one handler, may be rejected	CapturePayment	Queue to one service
Query	Request for data, synchronous	GET /payments/42	HTTP/gRPC

Event style	Payload	Use when	Cost
Notification	IDs only ({ orderId })	Consumers need fresh data or payload is sensitive/large	Consumers call back to the producer (coupling, load)
Event-carried state transfer	Relevant state ({ orderId, lines, total, address })	Consumers build local read models, producer must not be called	Larger payloads, schema discipline
Event sourcing	Events are the source of truth	Audit-heavy domains needing full history	High complexity; only with a clear need

Default: event-carried state transfer for integration events between services, with only the fields consumers need.

3. Envelope

{
  "id": "01J8Z4V7K1Q2ZP0M5N3R8T6W9X",
  "type": "orders.order_placed",
  "version": 2,
  "source": "orders-service",
  "occurredAt": "2026-03-14T10:22:31.512Z",
  "correlationId": "req-7f3a",
  "causationId": "cmd-91b2",
  "partitionKey": "order-4821",
  "data": { "orderId": "order-4821", "customerId": "cus-77", "totalCents": 12900, "currency": "EUR" }
}

CloudEvents is a reasonable standard envelope if the ecosystem uses it. Separate internal domain events (inside a service) from published integration events (a stable contract).

4. Transactional outbox

CREATE TABLE outbox (
  id            uuid PRIMARY KEY,
  aggregate_id  text NOT NULL,
  type          text NOT NULL,
  payload       jsonb NOT NULL,
  created_at    timestamptz NOT NULL DEFAULT now(),
  published_at  timestamptz
);
CREATE INDEX outbox_unpublished ON outbox (created_at) WHERE published_at IS NULL;

In the same transaction as the business change:

BEGIN;
UPDATE orders SET status = 'placed' WHERE id = $1;
INSERT INTO outbox (id, aggregate_id, type, payload) VALUES ($2, $1, 'orders.order_placed', $3);
COMMIT;

A relay publishes unpublished rows and marks them; or use CDC (Debezium reading the WAL) to stream the outbox table. Relay SQL, ordering caveats, and cleanup are in references/outbox-and-sagas.md; read it when implementing the relay or a saga.

5. Idempotent consumers

CREATE TABLE inbox (
  consumer   text NOT NULL,
  message_id uuid NOT NULL,
  processed_at timestamptz NOT NULL DEFAULT now(),
  PRIMARY KEY (consumer, message_id)
);

await db.transaction(async (tx) => {
  const inserted = await tx.query(
    'INSERT INTO inbox (consumer, message_id) VALUES ($1, $2) ON CONFLICT DO NOTHING RETURNING 1',
    ['shipping', msg.id]);
  if (inserted.rowCount === 0) return;          // duplicate: already processed
  await createShipment(tx, msg.data);           // side effects in the same transaction
});
await ack(msg);                                  // ack only after commit

• Alternatives: natural idempotence (upsert by business key, SET status = 'shipped') or version checks (WHERE version < $eventVersion) for state-transfer events.
• Side effects outside the DB (emails, third-party calls) need their own idempotency key (see idempotency-retries).
• Retry transient failures with backoff; after N attempts send to a dead-letter queue with the error and original headers. Alert on DLQ depth; provide a redrive tool.
• Poison messages must not block a partition forever.

6. Sagas

Style	How	Use when
Choreography	Each service reacts to events and emits its own	2–3 steps, few participants, simple compensation
Orchestration	A coordinator (state machine) sends commands and tracks state	4+ steps, branching, timeouts, need to see workflow state

Rules:

• Every step that can be undone has a compensating action (ReleaseStock for ReserveStock, RefundPayment for CapturePayment). Compensation is a new business action, not a rollback.
• Order steps so the hardest-to-compensate (sending goods, charging cards) comes last.
• Persist saga state; handle timeouts explicitly (e.g., cancel if payment not confirmed in 15 min).
• Workflow engines (Temporal, AWS Step Functions, and similar) are worth it when orchestration logic grows beyond a few states.

7. Schema evolution

Change	Compatible?	How
Add optional field	Yes (backward and forward)	Consumers ignore unknown fields
Add required field	No for old producers	Add as optional with default first
Remove field	No for consumers reading it	Deprecate, confirm no readers, then remove
Rename field	No	Add new, dual-write, migrate consumers, remove old
Change type/meaning	No	New event type or new major version
New enum value	Often breaks consumers with exhaustive switches	Document “ignore unknown values”; treat as breaking unless guaranteed

Use a schema registry (Avro/Protobuf/JSON Schema) with compatibility mode enforced in CI when available. For breaking changes, publish v2 alongside v1 until all consumers migrate.

Retention and replay: decide how long events are retained and whether consumers can rebuild state by replaying from the beginning. Replays must be idempotent too.

8. Observability

• Consumer lag per group/partition, DLQ depth, publish-to-consume latency (from occurredAt), outbox backlog age.
• Propagate correlationId and trace context in message headers so a request can be followed across services.

Gotchas

• Publishing inside the DB transaction before commit sends events for rolled-back data.
• Publishing after commit without an outbox loses events on crash.
• Consumers that call the producer synchronously for every notification recreate tight coupling with extra latency.
• Kafka consumer auto-commit can mark messages processed before the work finishes.
• Global ordering is almost never needed; when it seems needed, partition by the entity whose order matters.
• An event bus does not remove coupling: consumers depend on the event schema. Treat it as a public API.

Verification

1. Kill the service between DB commit and publish: the event still arrives (outbox works).
2. Deliver the same message twice: state changes once (inbox works).
3. Fail a middle saga step: compensations run and the system ends consistent.
4. Deploy a consumer on the old schema with a producer on the new one: no errors.