Visual Regression skill for Claude Code

A visual test is worth keeping only if it fails on real visual change and passes on every unchanged run, on every machine that runs it. Determinism comes first; thresholds are the last knob, not the first.

Ground rules

• Generate and compare baselines in the same environment: same OS image, browser version, fonts, and device scale factor. In practice, the Playwright Docker image.
• Remove nondeterminism at the source (frozen time, fixed data, disabled animations, loaded fonts) before raising thresholds.
• Snapshot meaningful states: a component or page section in a known state, not every page at every breakpoint.
• Baseline updates are code changes: reviewed in the PR with the diff images.
• Never run --update-snapshots blindly to make CI green.

Workflow

- [ ] 1. Decide scope: pages vs. components, browsers, viewports
- [ ] 2. Make the render deterministic
- [ ] 3. Write screenshot assertions with masks and stable names
- [ ] 4. Generate baselines inside the CI image
- [ ] 5. Run repeatedly to prove zero noise; tune thresholds last
- [ ] 6. Document the update workflow for the team

1. Scope

Need	Approach
Design system / component library	Component snapshots from Storybook stories (Playwright against Storybook, or a hosted service such as Chromatic)
Key marketing or app pages	Full-page or section snapshots in Playwright
Responsive layouts	2–3 viewports (e.g., 375, 768, 1280 wide) for layout-critical pages only
Cross-browser	Chromium by default; add WebKit/Firefox only for known rendering differences

Snapshot count drives maintenance cost. Start with ~20 high-value snapshots.

2. Deterministic rendering

// e2e/visual.setup.ts or inside a fixture
await page.clock.install({ time: new Date('2026-01-15T10:00:00Z') });
await page.route('**/api/dashboard', (route) => route.fulfill({ path: 'e2e/fixtures/dashboard.json' }));
await page.goto('/dashboard');
await page.evaluate(async () => { await document.fonts.ready; });

Checklist of noise sources:

• Time: relative dates (“3 minutes ago”), clocks, countdowns → page.clock.
• Data: random IDs, avatars, ads, A/B variants → fixture responses; disable experiments via flag or cookie.
• Animations and transitions: toHaveScreenshot disables CSS animations by default (animations: 'disabled'); JS-driven animation (Lottie, canvas, carousels) needs to be paused or masked.
• Fonts: self-host fonts in the app or fixtures; wait for document.fonts.ready. Missing fonts in the container are the top cause of cross-machine diffs.
• Lazy images: scroll or force-load, then wait for img.complete, or mask them.
• Caret and focus rings: caret is hidden by default; blur focused inputs.
• Scrollbars: differ by OS; hide via injected CSS.
• Third-party embeds (maps, video, chat widgets) → mask or block.

Inject a stylesheet for test-only stabilization:

/* e2e/screenshot.css */
*, *::before, *::after { transition: none !important; animation: none !important; }
::-webkit-scrollbar { display: none; }
[data-visual-ignore] { visibility: hidden !important; }

3. Assertions

import { test, expect } from '@playwright/test';

test('pricing page', async ({ page }) => {
  await page.goto('/pricing');
  await expect(page.getByRole('heading', { name: 'Pricing' })).toBeVisible();
  await expect(page).toHaveScreenshot('pricing.png', {
    fullPage: true,
    mask: [page.getByTestId('live-visitor-count')],
    stylePath: 'e2e/screenshot.css',
  });
});

test('plan card, highlighted state', async ({ page }) => {
  await page.goto('/pricing');
  const card = page.getByRole('article', { name: 'Team plan' });
  await card.hover();
  await expect(card).toHaveScreenshot('plan-card-hover.png');
});

• Wait for a meaningful visible state before the screenshot; toHaveScreenshot also retries until two consecutive captures match, which absorbs late paints.
• Element screenshots beat full pages: smaller diffs, fewer unrelated failures.
• Name snapshots explicitly; default names depend on test titles and break on renames.

Global defaults in config:

// playwright.config.ts
export default defineConfig({
  expect: {
    toHaveScreenshot: { maxDiffPixelRatio: 0.002, threshold: 0.2, animations: 'disabled' },
  },
  snapshotPathTemplate: '{testDir}/__screenshots__/{testFilePath}/{arg}-{projectName}{ext}',
});

The default template includes the platform (-darwin, -linux) in the name. The template above drops it, which is correct only when every run (local and CI) happens in the same Linux container. Keep the platform suffix if developers generate baselines natively.

4. Baselines in the CI image

# Match the tag to the installed @playwright/test version (npm ls @playwright/test)
docker run --rm --ipc=host -v "$PWD":/work -w /work \
  mcr.microsoft.com/playwright:v1.56.0-noble \
  /bin/bash -c "npm ci && npx playwright test --project=visual --update-snapshots"

Run the same image in CI. Developers on macOS/Windows update baselines through this command (wrap it in an npm script like test:visual:update), never natively.

Recent Playwright versions accept a mode on --update-snapshots (for example missing or changed); check npx playwright test --help for the installed version before relying on it.

5. Thresholds

Knob	Meaning	Start at
threshold	Per-pixel color distance tolerance (0–1, YIQ)	0.2 (default)
maxDiffPixels	Absolute number of differing pixels allowed	0–100 for elements
maxDiffPixelRatio	Fraction of differing pixels allowed	0.001–0.005 for pages

Prove noise is zero first: run the visual project 10 times in the container (--repeat-each=10). If any run diffs, fix the source of nondeterminism. Raise thresholds only for known sub-pixel anti-aliasing differences, per snapshot rather than globally, and leave a comment saying why.

6. Reviewing and updating

• On failure, Playwright writes -expected, -actual, and -diff images to test-results/; the HTML report shows a slider comparison. Upload them as CI artifacts.
• Intentional change: update baselines in the container on the PR branch, commit them, and let reviewers see the image diff in the PR.
• Large baselines bloat the repo; consider Git LFS for __screenshots__/ when snapshot volume grows past a few hundred files.

Gotchas

• Baselines generated on a Mac and compared on Linux CI always fail (font rendering).
• GPU-dependent rendering (WebGL, heavy filters) differs between headless runs; mask the canvas or snapshot with software rendering consistently.
• Viewport and deviceScaleFactor must be fixed in the project config.
• A diff that only appears in CI and not in the container locally usually means a different image tag or missing font package.
• Visual tests do not replace accessibility or functional assertions; a pixel-perfect button can still be unclickable.

Verification

1. Ten repeated runs in the CI image produce zero diffs.
2. A deliberate CSS change (1px padding on a key component) fails exactly the expected snapshots and no others.
3. CI uploads expected/actual/diff images on failure.