Coverage Analysis skill for Claude Code

Find untested risk instead of chasing a percentage. Claude fixes how coverage is measured first (branch mode, all source files, merged shards), then reads the report for risk by combining missed lines with churn and business criticality. It gates pull requests on coverage of changed lines rather than the global number, and flags tests that execute code without asserting anything. A bundled script computes diff coverage from LCOV or Cobertura.

  • namecoverage-analysis
  • size189 lines · ≈2,350 tokens
  • files2
  • licenseMIT
  • updated

What Claude sees before loading it

This description is the only part of the skill that sits in context all the time. Claude reads it to decide when to load the full playbook below.

description: Uses code coverage as a risk-finding tool rather than a vanity number — line vs branch coverage, measuring all source files instead of only loaded ones, diff (patch) coverage gates on pull requests, ranking untested risk by churn and criticality, and spotting coverage theater such as assertion-free tests. Covers Istanbul/V8, coverage.py, JaCoCo, Go cover, and cargo-llvm-cov, with a bundled LCOV/Cobertura diff-coverage script. Use when setting or debating a coverage threshold, adding a CI coverage gate, reading a coverage report, or asking where tests are missing.

Try asking

  • Our coverage is 85% but bugs keep shipping. Where are the real gaps?
  • Set up a PR check that fails when new code is under 80% covered
  • Should we raise the global coverage threshold to 90%? Look at the report and advise

Use it when

  • Choosing or arguing about a coverage threshold
  • Adding a coverage gate to CI or Codecov/SonarQube
  • Deciding where to write tests next in a legacy codebase
  • A coverage number looks suspiciously high or jumps between runs

Skip it when

  • You need to know whether covered code is actually checked by assertions: use mutation-testing
  • The question is how to write a particular test well: use unit-testing

How to check the result

Deleting one test drops coverage for exactly the lines it exercised, a new untested file shows at 0%, and the local diff-coverage run matches the CI gate result.

The common mistake it prevents

Leaving files that no test imports out of the report, so untested modules are invisible and the percentage looks healthier than the codebase is.

~/.claude/skills/coverage-analysis/SKILL.md

--- name: coverage-analysis description: Uses code coverage as a risk-finding tool rather than a vanity number — line vs branch coverage, measuring all source files instead of only… license: MIT ---

Coverage tells you what the tests never executed; it says nothing about whether executed code was checked. Done well means: every source file is measured, branch coverage is on, the PR gate is on the diff rather than the global number, and the report is read to find risky untested code, not to hit a percentage.

Ground rules

  • Gate new code, not old code. A diff-coverage gate (e.g. 80% of changed executable lines) improves the codebase steadily; a global threshold blocks unrelated PRs and invites filler tests.
  • Turn branch coverage on everywhere it is optional. Line coverage marks return a ? b : c covered when only one side ran.
  • Measure all files. Most tools only report files a test imported; an untested module is invisible and inflates the percentage.
  • Never write a test whose only purpose is to execute lines. A test without an assertion that could fail is coverage theater; mutation testing exposes it.
  • Exclusions need a reason in the code (/* istanbul ignore next -- generated */, # pragma: no cover - platform specific). Blanket directory excludes hide real code.
  • Do not chase 100%. Typical healthy targets: 80–90% branch on domain logic, lower on glue, UI wiring and generated code. Critical paths (money, auth, data deletion) get reviewed line by line regardless of the number.

Workflow

- [ ] 1. Find the current coverage tool, config, and where reports land
- [ ] 2. Fix measurement: all files, branch mode, correct excludes, merged reports
- [ ] 3. Produce LCOV or Cobertura output in CI
- [ ] 4. Read the report for risk (worst files x criticality x churn)
- [ ] 5. Add the diff-coverage gate on PRs; keep global as a ratchet, not a gate
- [ ] 6. For key modules, check test strength (mutation testing, assertion review)
- [ ] 7. Report findings with the template below

1. Inspect first

grep -nE '"(test|coverage)[^"]*"' package.json
ls jest.config.* vitest.config.* .nycrc* .c8rc* pyproject.toml setup.cfg .coveragerc \
   build.gradle* pom.xml codecov.yml .codecov.yml sonar-project.properties 2>/dev/null
grep -rnE 'coverage|codecov|coveralls|sonar' .github/workflows 2>/dev/null | head -30

Note: which provider (V8 or Istanbul), whether branches are measured, include/exclude patterns, current thresholds, and whether multiple test jobs (unit, integration, shards) upload separate reports that must be merged.

2. Fix measurement per stack

StackBranches + all filesOutput for tooling
Vitestcoverage: { provider: 'v8', include: ['src/**'] } (branches reported by default)reporter: ['text', 'lcov', 'cobertura']
JestcollectCoverageFrom: ['src/**/*.{ts,tsx}', '!src/**/*.d.ts'], coverageProvider: 'v8' or 'babel'coverageReporters: ['text-summary', 'lcov', 'cobertura']
coverage.py / pytest-cov[tool.coverage.run] branch = true, source = ["src/pkg"]pytest --cov --cov-branch --cov-report=xml --cov-report=term-missing
Gogo test -covermode=atomic -coverpkg=./... -coverprofile=cover.out ./... (statement only; no branch mode)go tool cover -func=cover.out; convert with gocover-cobertura if needed
JVMJaCoCo plugin; read BRANCH counter, not just LINEjacocoTestReport XML
Rustcargo llvm-cov --lcov --output-path lcov.info (--branch needs nightly)LCOV
.NETdotnet test --collect:"XPlat Code Coverage"Cobertura

Without include / collectCoverageFrom / source, files never imported by a test do not appear at all. Without -coverpkg=./..., Go only counts the package under test.

Merging: combine shards and suites before judging (coverage combine for Python with parallel = true; nyc merge or istanbul-merge for Istanbul JSON; lcov -a a.info -a b.info -o all.info; Codecov merges uploads per commit automatically).

3. Read the report for risk

python3 scripts/coverage_report.py worst coverage/lcov.info --top 20
python3 scripts/coverage_report.py worst coverage.xml --top 20      # Cobertura works too

The script ranks files by missed lines (absolute untested volume), shows line and branch percentages, and accepts LCOV or Cobertura. Then weight by risk:

SignalHow to get itWhy
Missed lines/branchesworst outputVolume of unexercised code
Churngit log --since=6.months --name-only --format= | sort | uniq -c | sort -rn | head -30Frequently changed + untested = where regressions come from
CriticalityPaths touching money, auth, permissions, deletion, migrationsCost of a bug
Bug historygit log -i -E --grep='fix|bug' --name-only --format=Past failures cluster

Prioritize files high on two or more signals. A 40%-covered admin CSV export matters less than a 75%-covered refund calculator with uncovered error branches.

What to look for inside a file: uncovered catch/error branches, else arms of validation, early returns for edge inputs, feature-flag off paths, retry/timeout paths. These are the paths production hits on bad days.

4. Diff coverage gate on PRs

git fetch origin main
git diff -U0 origin/main...HEAD -- . ':!**/*.test.*' > change.diff
python3 scripts/coverage_report.py diff coverage/lcov.info change.diff --fail-under 80

The script lists each changed file with changed executable lines, missed lines as ranges, and lines whose branches were only partially taken; it exits 1 below the threshold and names changed files absent from the report (unmeasured, which means the measurement config is wrong, not that they are fine).

Hosted equivalents: Codecov coverage.status.patch (target 80%, threshold for noise), diff-cover coverage.xml --compare-branch=origin/main --fail-under=80 (Python package), SonarQube “coverage on new code” quality gate. Pick one; do not stack three gates.

Global number policy: report it, ratchet it (fail only if it drops by more than ~0.5–1 point versus main), never use it as an absolute PR gate on an existing codebase.

5. Check test strength where it matters

Coverage high but bugs still ship → executed but unchecked code. For the top-risk modules:

  • Grep for tests without assertions: grep -L -E 'expect|assert|should' $(git ls-files '*test*').
  • Look for snapshot-only tests of logic, toBeDefined()/toBeTruthy() as the sole assertion, and tests that catch and swallow exceptions.
  • Run mutation testing scoped to those files (see mutation-testing). Surviving mutants in covered lines are the real gap.

Coverage theater — reject in review

PatternWhy it is theater
Test calls a function and asserts nothingExecutes lines, verifies nothing
expect(result).toBeDefined() on a computed valuePasses for nearly any bug
Testing private helpers directly to bump numbersCouples tests to structure, no behavior gain
Excluding hard files from coverage to pass a gateHides exactly the risky code
Snapshot of a large object as the only checkGets blindly updated with -u
Lowering the threshold in the same PR that drops coverageGate becomes decoration

Gotchas

  • V8 and Istanbul count differently (V8 measures byte ranges, Istanbul instruments statements); switching provider moves the number a few points without any test change. Re-baseline when switching.
  • Source maps: TS coverage through bundlers can attribute lines to the wrong file if source maps are off; check that report paths point to .ts sources.
  • Path mismatch: CI reports often contain absolute paths (/home/runner/work/...). Hosted tools fix paths by suffix; the bundled script matches by unique suffix too. If a changed file is listed as “not in coverage report”, inspect SF: lines in the LCOV.
  • Generated code, migrations and vendored files belong in excludes; config files with logic (feature flags, routing tables) do not.
  • Integration and E2E coverage are useful for finding dead code but should not be merged into the unit gate unless every PR runs them; otherwise the number swings by job.
  • Go has no branch coverage; compensate by table-driven tests that name each branch.
  • 100% line coverage with branch coverage off can still miss half the decision outcomes.

Verification

  1. Delete one test for a known file and re-run: the file’s coverage drops and the uncovered lines are the ones that test exercised. If nothing changes, measurement is misconfigured.
  2. Add an untested new file in a scratch branch: it must appear at 0%, and the diff gate must fail.
  3. Run the diff gate locally against origin/main and confirm it matches the CI result.
  4. For theater checks, mutate one covered line by hand (flip a comparison); at least one test must fail.

Output template

## Coverage findings — <repo> @ <commit>

Measurement: <tool/provider>, branches on/off, all-files on/off, merged from <jobs>
Totals: lines X%, branches Y% (do not gate on these)

### Highest risk untested code
| File | Missed lines | Branch % | Churn (6 mo) | Why it matters |
|---|---|---|---|---|

### Specific gaps
- <file>:<lines> — <error/else/edge path> not exercised; suggested test: <behavior>

### Theater / weak tests
- <test file> — <pattern>, recommend <fix>

### Gate recommendation
Diff coverage >= <N>% on changed lines; global ratchet tolerance <x> points; excludes: <list with reasons>

Bundled files

Loaded only when a step in the skill needs them. Review them here before installing.

scripts/coverage_report.py 212 lines · 7.9 KB
#!/usr/bin/env python3
"""Coverage triage from LCOV or Cobertura XML, standard library only.

Subcommands:
  diff   Coverage of the lines a unified diff adds (diff coverage), with
         uncovered line ranges per file. Exits 1 if below --fail-under.
  worst  Files ranked by missed lines (where untested code actually is).

Examples:
  git diff -U0 origin/main...HEAD > change.diff
  python3 coverage_report.py diff coverage/lcov.info change.diff --fail-under 80
  python3 coverage_report.py worst coverage.xml --top 15
"""
from __future__ import annotations

import argparse
import re
import sys
import xml.etree.ElementTree as ET
from collections import defaultdict
from pathlib import PurePosixPath


class FileCov:
    def __init__(self) -> None:
        self.lines: dict[int, int] = {}          # line -> hit count
        self.branches: dict[int, list[bool]] = defaultdict(list)  # line -> taken flags

    def add_line(self, n: int, hits: int) -> None:
        self.lines[n] = max(self.lines.get(n, 0), hits)


def norm(path: str) -> str:
    return str(PurePosixPath(path.replace("\\", "/"))).lstrip("./")


def parse_lcov(text: str) -> dict[str, FileCov]:
    files: dict[str, FileCov] = {}
    cur: FileCov | None = None
    for raw in text.splitlines():
        line = raw.strip()
        if line.startswith("SF:"):
            cur = files.setdefault(norm(line[3:]), FileCov())
        elif cur is None:
            continue
        elif line.startswith("DA:"):
            parts = line[3:].split(",")
            cur.add_line(int(parts[0]), int(float(parts[1])))
        elif line.startswith("BRDA:"):
            ln, _blk, _br, taken = line[5:].split(",")
            cur.branches[int(ln)].append(taken not in ("-", "0"))
        elif line == "end_of_record":
            cur = None
    return files


def parse_cobertura(text: str) -> dict[str, FileCov]:
    root = ET.fromstring(text)
    sources = [s.text.strip() for s in root.iter("source") if s.text and s.text.strip()]
    files: dict[str, FileCov] = {}
    for cls in root.iter("class"):
        fname = cls.get("filename", "")
        fc = files.setdefault(norm(fname), FileCov())
        for ln in cls.iter("line"):
            n, hits = int(ln.get("number", 0)), int(float(ln.get("hits", 0)))
            fc.add_line(n, hits)
            cond = ln.get("condition-coverage")  # e.g. "50% (1/2)"
            if ln.get("branch") == "true" and cond:
                m = re.search(r"\((\d+)/(\d+)\)", cond)
                if m:
                    got, tot = int(m.group(1)), int(m.group(2))
                    fc.branches[n] = [True] * got + [False] * (tot - got)
    _ = sources  # paths in Cobertura are relative to <source>; suffix matching covers both
    return files


def load(path: str) -> dict[str, FileCov]:
    with open(path, encoding="utf-8") as fh:
        text = fh.read()
    if text.lstrip().startswith("<"):
        return parse_cobertura(text)
    return parse_lcov(text)


def parse_diff(text: str) -> dict[str, set[int]]:
    added: dict[str, set[int]] = defaultdict(set)
    cur: str | None = None
    new_line = 0
    for line in text.splitlines():
        if line.startswith("+++ "):
            target = line[4:].split("\t")[0]
            cur = None if target == "/dev/null" else norm(re.sub(r"^b/", "", target))
        elif line.startswith("@@"):
            m = re.match(r"@@ -\d+(?:,\d+)? \+(\d+)(?:,\d+)? @@", line)
            new_line = int(m.group(1)) if m else 0
        elif cur is None or line.startswith("--- "):
            continue
        elif line.startswith("+"):
            added[cur].add(new_line)
            new_line += 1
        elif line.startswith("-") or line.startswith("\\"):
            continue
        else:
            new_line += 1
    return added


def match(cov: dict[str, FileCov], diff_path: str) -> FileCov | None:
    if diff_path in cov:
        return cov[diff_path]
    hits = [k for k in cov if k.endswith("/" + diff_path) or diff_path.endswith("/" + k)]
    return cov[hits[0]] if len(hits) == 1 else None


def ranges(nums: list[int]) -> str:
    out, start, prev = [], None, None
    for n in sorted(nums):
        if start is None:
            start = prev = n
        elif n == prev + 1:
            prev = n
        else:
            out.append(f"{start}" if start == prev else f"{start}-{prev}")
            start = prev = n
    if start is not None:
        out.append(f"{start}" if start == prev else f"{start}-{prev}")
    return ",".join(out)


def cmd_diff(args: argparse.Namespace) -> int:
    cov = load(args.coverage)
    with open(args.diff, encoding="utf-8") as fh:
        added = parse_diff(fh.read())
    total = covered = 0
    unmatched = []
    rows = []
    for path, lines in sorted(added.items()):
        fc = match(cov, path)
        if fc is None:
            unmatched.append(path)
            continue
        exe = [n for n in lines if n in fc.lines]
        if not exe:
            continue
        miss = [n for n in exe if fc.lines[n] == 0]
        partial = [n for n in exe if n in fc.branches and not all(fc.branches[n]) and fc.lines[n] > 0]
        total += len(exe)
        covered += len(exe) - len(miss)
        rows.append((path, len(exe), len(miss), miss, partial))
    print(f"{'file':60} {'changed':>7} {'missed':>6}  uncovered lines / partial branches")
    for path, n, m, miss, partial in sorted(rows, key=lambda r: -r[2]):
        extra = f"  partial: {ranges(partial)}" if partial else ""
        print(f"{path[-60:]:60} {n:7d} {m:6d}  {ranges(miss) or '-'}{extra}")
    if unmatched:
        print(f"\nnot in coverage report (unmeasured, not 'covered'): {', '.join(unmatched)}")
    if total == 0:
        print("\nno executable changed lines found in the coverage report")
        return 0
    pct = 100.0 * covered / total
    print(f"\ndiff coverage: {covered}/{total} executable changed lines = {pct:.1f}%")
    if args.fail_under is not None and pct < args.fail_under:
        print(f"FAIL: below threshold {args.fail_under:.1f}%")
        return 1
    return 0


def cmd_worst(args: argparse.Namespace) -> int:
    cov = load(args.coverage)
    rows = []
    t_lines = t_hit = t_br = t_br_hit = 0
    for path, fc in cov.items():
        n = len(fc.lines)
        if n == 0:
            continue
        hit = sum(1 for h in fc.lines.values() if h > 0)
        br = sum(len(v) for v in fc.branches.values())
        br_hit = sum(sum(v) for v in fc.branches.values())
        t_lines, t_hit, t_br, t_br_hit = t_lines + n, t_hit + hit, t_br + br, t_br_hit + br_hit
        rows.append((path, n, n - hit, 100.0 * hit / n, (100.0 * br_hit / br) if br else None))
    rows.sort(key=lambda r: (-r[2], r[3]))
    print(f"{'file':60} {'lines':>6} {'missed':>6} {'line%':>6} {'branch%':>7}")
    for path, n, miss, lp, bp in rows[: args.top]:
        bps = f"{bp:6.1f}%" if bp is not None else "     -"
        print(f"{path[-60:]:60} {n:6d} {miss:6d} {lp:5.1f}% {bps}")
    if t_lines:
        bsum = f", branches {100.0 * t_br_hit / t_br:.1f}%" if t_br else ""
        print(f"\ntotal: lines {100.0 * t_hit / t_lines:.1f}% ({t_hit}/{t_lines}){bsum}")
    return 0


def main() -> int:
    p = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
    sub = p.add_subparsers(dest="cmd", required=True)
    d = sub.add_parser("diff", help="coverage of lines added by a unified diff")
    d.add_argument("coverage", help="lcov.info or Cobertura coverage.xml")
    d.add_argument("diff", help="unified diff file (git diff -U0 base...HEAD)")
    d.add_argument("--fail-under", type=float, default=None)
    d.set_defaults(func=cmd_diff)
    w = sub.add_parser("worst", help="files ranked by missed lines")
    w.add_argument("coverage")
    w.add_argument("--top", type=int, default=20)
    w.set_defaults(func=cmd_worst)
    args = p.parse_args()
    try:
        return args.func(args)
    except (OSError, ET.ParseError, ValueError) as exc:
        print(f"error: {exc}", file=sys.stderr)
        return 2


if __name__ == "__main__":
    sys.exit(main())

About this skill

Written and maintained by the Claude Skills editors to the same editorial standard as every skill in the directory: a selective trigger, a real workflow, known pitfalls and a verification step. First published , last updated . MIT licensed.

esc