Coverage Analysis skill for Claude Code
Find untested risk instead of chasing a percentage. Claude fixes how coverage is measured first (branch mode, all source files, merged shards), then reads the report for risk by combining missed lines with churn and business criticality. It gates pull requests on coverage of changed lines rather than the global number, and flags tests that execute code without asserting anything. A bundled script computes diff coverage from LCOV or Cobertura.
- name
coverage-analysis - size189 lines · ≈2,350 tokens
- files2
- licenseMIT
- updated
What Claude sees before loading it
This description is the only part of the skill that sits in context all the time. Claude reads it to decide when to load the full playbook below.
description: Uses code coverage as a risk-finding tool rather than a vanity number — line vs branch coverage, measuring all source files instead of only loaded ones, diff (patch) coverage gates on pull requests, ranking untested risk by churn and criticality, and spotting coverage theater such as assertion-free tests. Covers Istanbul/V8, coverage.py, JaCoCo, Go cover, and cargo-llvm-cov, with a bundled LCOV/Cobertura diff-coverage script. Use when setting or debating a coverage threshold, adding a CI coverage gate, reading a coverage report, or asking where tests are missing. Try asking
- Our coverage is 85% but bugs keep shipping. Where are the real gaps?
- Set up a PR check that fails when new code is under 80% covered
- Should we raise the global coverage threshold to 90%? Look at the report and advise
Use it when
- Choosing or arguing about a coverage threshold
- Adding a coverage gate to CI or Codecov/SonarQube
- Deciding where to write tests next in a legacy codebase
- A coverage number looks suspiciously high or jumps between runs
Skip it when
- You need to know whether covered code is actually checked by assertions: use mutation-testing
- The question is how to write a particular test well: use unit-testing
How to check the result
Deleting one test drops coverage for exactly the lines it exercised, a new untested file shows at 0%, and the local diff-coverage run matches the CI gate result.
The common mistake it prevents
Leaving files that no test imports out of the report, so untested modules are invisible and the percentage looks healthier than the codebase is.
Coverage tells you what the tests never executed; it says nothing about whether executed code was checked. Done well means: every source file is measured, branch coverage is on, the PR gate is on the diff rather than the global number, and the report is read to find risky untested code, not to hit a percentage.
Ground rules
- Gate new code, not old code. A diff-coverage gate (e.g. 80% of changed executable lines) improves the codebase steadily; a global threshold blocks unrelated PRs and invites filler tests.
- Turn branch coverage on everywhere it is optional. Line coverage marks
return a ? b : ccovered when only one side ran. - Measure all files. Most tools only report files a test imported; an untested module is invisible and inflates the percentage.
- Never write a test whose only purpose is to execute lines. A test without an assertion that could fail is coverage theater; mutation testing exposes it.
- Exclusions need a reason in the code (
/* istanbul ignore next -- generated */,# pragma: no cover - platform specific). Blanket directory excludes hide real code. - Do not chase 100%. Typical healthy targets: 80–90% branch on domain logic, lower on glue, UI wiring and generated code. Critical paths (money, auth, data deletion) get reviewed line by line regardless of the number.
Workflow
- [ ] 1. Find the current coverage tool, config, and where reports land
- [ ] 2. Fix measurement: all files, branch mode, correct excludes, merged reports
- [ ] 3. Produce LCOV or Cobertura output in CI
- [ ] 4. Read the report for risk (worst files x criticality x churn)
- [ ] 5. Add the diff-coverage gate on PRs; keep global as a ratchet, not a gate
- [ ] 6. For key modules, check test strength (mutation testing, assertion review)
- [ ] 7. Report findings with the template below
1. Inspect first
grep -nE '"(test|coverage)[^"]*"' package.json
ls jest.config.* vitest.config.* .nycrc* .c8rc* pyproject.toml setup.cfg .coveragerc \
build.gradle* pom.xml codecov.yml .codecov.yml sonar-project.properties 2>/dev/null
grep -rnE 'coverage|codecov|coveralls|sonar' .github/workflows 2>/dev/null | head -30
Note: which provider (V8 or Istanbul), whether branches are measured, include/exclude patterns, current thresholds, and whether multiple test jobs (unit, integration, shards) upload separate reports that must be merged.
2. Fix measurement per stack
| Stack | Branches + all files | Output for tooling |
|---|---|---|
| Vitest | coverage: { provider: 'v8', include: ['src/**'] } (branches reported by default) | reporter: ['text', 'lcov', 'cobertura'] |
| Jest | collectCoverageFrom: ['src/**/*.{ts,tsx}', '!src/**/*.d.ts'], coverageProvider: 'v8' or 'babel' | coverageReporters: ['text-summary', 'lcov', 'cobertura'] |
| coverage.py / pytest-cov | [tool.coverage.run] branch = true, source = ["src/pkg"] | pytest --cov --cov-branch --cov-report=xml --cov-report=term-missing |
| Go | go test -covermode=atomic -coverpkg=./... -coverprofile=cover.out ./... (statement only; no branch mode) | go tool cover -func=cover.out; convert with gocover-cobertura if needed |
| JVM | JaCoCo plugin; read BRANCH counter, not just LINE | jacocoTestReport XML |
| Rust | cargo llvm-cov --lcov --output-path lcov.info (--branch needs nightly) | LCOV |
| .NET | dotnet test --collect:"XPlat Code Coverage" | Cobertura |
Without include / collectCoverageFrom / source, files never imported by a test do
not appear at all. Without -coverpkg=./..., Go only counts the package under test.
Merging: combine shards and suites before judging (coverage combine for Python with
parallel = true; nyc merge or istanbul-merge for Istanbul JSON; lcov -a a.info -a b.info -o all.info; Codecov merges uploads per commit automatically).
3. Read the report for risk
python3 scripts/coverage_report.py worst coverage/lcov.info --top 20
python3 scripts/coverage_report.py worst coverage.xml --top 20 # Cobertura works too
The script ranks files by missed lines (absolute untested volume), shows line and branch percentages, and accepts LCOV or Cobertura. Then weight by risk:
| Signal | How to get it | Why |
|---|---|---|
| Missed lines/branches | worst output | Volume of unexercised code |
| Churn | git log --since=6.months --name-only --format= | sort | uniq -c | sort -rn | head -30 | Frequently changed + untested = where regressions come from |
| Criticality | Paths touching money, auth, permissions, deletion, migrations | Cost of a bug |
| Bug history | git log -i -E --grep='fix|bug' --name-only --format= | Past failures cluster |
Prioritize files high on two or more signals. A 40%-covered admin CSV export matters less than a 75%-covered refund calculator with uncovered error branches.
What to look for inside a file: uncovered catch/error branches, else arms of
validation, early returns for edge inputs, feature-flag off paths, retry/timeout paths.
These are the paths production hits on bad days.
4. Diff coverage gate on PRs
git fetch origin main
git diff -U0 origin/main...HEAD -- . ':!**/*.test.*' > change.diff
python3 scripts/coverage_report.py diff coverage/lcov.info change.diff --fail-under 80
The script lists each changed file with changed executable lines, missed lines as ranges, and lines whose branches were only partially taken; it exits 1 below the threshold and names changed files absent from the report (unmeasured, which means the measurement config is wrong, not that they are fine).
Hosted equivalents: Codecov coverage.status.patch (target 80%, threshold for noise),
diff-cover coverage.xml --compare-branch=origin/main --fail-under=80 (Python package),
SonarQube “coverage on new code” quality gate. Pick one; do not stack three gates.
Global number policy: report it, ratchet it (fail only if it drops by more than ~0.5–1 point versus main), never use it as an absolute PR gate on an existing codebase.
5. Check test strength where it matters
Coverage high but bugs still ship → executed but unchecked code. For the top-risk modules:
- Grep for tests without assertions:
grep -L -E 'expect|assert|should' $(git ls-files '*test*'). - Look for snapshot-only tests of logic,
toBeDefined()/toBeTruthy()as the sole assertion, and tests that catch and swallow exceptions. - Run mutation testing scoped to those files (see mutation-testing). Surviving mutants in covered lines are the real gap.
Coverage theater — reject in review
| Pattern | Why it is theater |
|---|---|
| Test calls a function and asserts nothing | Executes lines, verifies nothing |
expect(result).toBeDefined() on a computed value | Passes for nearly any bug |
| Testing private helpers directly to bump numbers | Couples tests to structure, no behavior gain |
| Excluding hard files from coverage to pass a gate | Hides exactly the risky code |
| Snapshot of a large object as the only check | Gets blindly updated with -u |
| Lowering the threshold in the same PR that drops coverage | Gate becomes decoration |
Gotchas
- V8 and Istanbul count differently (V8 measures byte ranges, Istanbul instruments statements); switching provider moves the number a few points without any test change. Re-baseline when switching.
- Source maps: TS coverage through bundlers can attribute lines to the wrong file if
source maps are off; check that report paths point to
.tssources. - Path mismatch: CI reports often contain absolute paths (
/home/runner/work/...). Hosted tools fix paths by suffix; the bundled script matches by unique suffix too. If a changed file is listed as “not in coverage report”, inspectSF:lines in the LCOV. - Generated code, migrations and vendored files belong in excludes; config files with logic (feature flags, routing tables) do not.
- Integration and E2E coverage are useful for finding dead code but should not be merged into the unit gate unless every PR runs them; otherwise the number swings by job.
- Go has no branch coverage; compensate by table-driven tests that name each branch.
- 100% line coverage with branch coverage off can still miss half the decision outcomes.
Verification
- Delete one test for a known file and re-run: the file’s coverage drops and the uncovered lines are the ones that test exercised. If nothing changes, measurement is misconfigured.
- Add an untested new file in a scratch branch: it must appear at 0%, and the diff gate must fail.
- Run the diff gate locally against
origin/mainand confirm it matches the CI result. - For theater checks, mutate one covered line by hand (flip a comparison); at least one test must fail.
Output template
## Coverage findings — <repo> @ <commit>
Measurement: <tool/provider>, branches on/off, all-files on/off, merged from <jobs>
Totals: lines X%, branches Y% (do not gate on these)
### Highest risk untested code
| File | Missed lines | Branch % | Churn (6 mo) | Why it matters |
|---|---|---|---|---|
### Specific gaps
- <file>:<lines> — <error/else/edge path> not exercised; suggested test: <behavior>
### Theater / weak tests
- <test file> — <pattern>, recommend <fix>
### Gate recommendation
Diff coverage >= <N>% on changed lines; global ratchet tolerance <x> points; excludes: <list with reasons> Bundled files
Loaded only when a step in the skill needs them. Review them here before installing.
scripts/coverage_report.py
#!/usr/bin/env python3
"""Coverage triage from LCOV or Cobertura XML, standard library only.
Subcommands:
diff Coverage of the lines a unified diff adds (diff coverage), with
uncovered line ranges per file. Exits 1 if below --fail-under.
worst Files ranked by missed lines (where untested code actually is).
Examples:
git diff -U0 origin/main...HEAD > change.diff
python3 coverage_report.py diff coverage/lcov.info change.diff --fail-under 80
python3 coverage_report.py worst coverage.xml --top 15
"""
from __future__ import annotations
import argparse
import re
import sys
import xml.etree.ElementTree as ET
from collections import defaultdict
from pathlib import PurePosixPath
class FileCov:
def __init__(self) -> None:
self.lines: dict[int, int] = {} # line -> hit count
self.branches: dict[int, list[bool]] = defaultdict(list) # line -> taken flags
def add_line(self, n: int, hits: int) -> None:
self.lines[n] = max(self.lines.get(n, 0), hits)
def norm(path: str) -> str:
return str(PurePosixPath(path.replace("\\", "/"))).lstrip("./")
def parse_lcov(text: str) -> dict[str, FileCov]:
files: dict[str, FileCov] = {}
cur: FileCov | None = None
for raw in text.splitlines():
line = raw.strip()
if line.startswith("SF:"):
cur = files.setdefault(norm(line[3:]), FileCov())
elif cur is None:
continue
elif line.startswith("DA:"):
parts = line[3:].split(",")
cur.add_line(int(parts[0]), int(float(parts[1])))
elif line.startswith("BRDA:"):
ln, _blk, _br, taken = line[5:].split(",")
cur.branches[int(ln)].append(taken not in ("-", "0"))
elif line == "end_of_record":
cur = None
return files
def parse_cobertura(text: str) -> dict[str, FileCov]:
root = ET.fromstring(text)
sources = [s.text.strip() for s in root.iter("source") if s.text and s.text.strip()]
files: dict[str, FileCov] = {}
for cls in root.iter("class"):
fname = cls.get("filename", "")
fc = files.setdefault(norm(fname), FileCov())
for ln in cls.iter("line"):
n, hits = int(ln.get("number", 0)), int(float(ln.get("hits", 0)))
fc.add_line(n, hits)
cond = ln.get("condition-coverage") # e.g. "50% (1/2)"
if ln.get("branch") == "true" and cond:
m = re.search(r"\((\d+)/(\d+)\)", cond)
if m:
got, tot = int(m.group(1)), int(m.group(2))
fc.branches[n] = [True] * got + [False] * (tot - got)
_ = sources # paths in Cobertura are relative to <source>; suffix matching covers both
return files
def load(path: str) -> dict[str, FileCov]:
with open(path, encoding="utf-8") as fh:
text = fh.read()
if text.lstrip().startswith("<"):
return parse_cobertura(text)
return parse_lcov(text)
def parse_diff(text: str) -> dict[str, set[int]]:
added: dict[str, set[int]] = defaultdict(set)
cur: str | None = None
new_line = 0
for line in text.splitlines():
if line.startswith("+++ "):
target = line[4:].split("\t")[0]
cur = None if target == "/dev/null" else norm(re.sub(r"^b/", "", target))
elif line.startswith("@@"):
m = re.match(r"@@ -\d+(?:,\d+)? \+(\d+)(?:,\d+)? @@", line)
new_line = int(m.group(1)) if m else 0
elif cur is None or line.startswith("--- "):
continue
elif line.startswith("+"):
added[cur].add(new_line)
new_line += 1
elif line.startswith("-") or line.startswith("\\"):
continue
else:
new_line += 1
return added
def match(cov: dict[str, FileCov], diff_path: str) -> FileCov | None:
if diff_path in cov:
return cov[diff_path]
hits = [k for k in cov if k.endswith("/" + diff_path) or diff_path.endswith("/" + k)]
return cov[hits[0]] if len(hits) == 1 else None
def ranges(nums: list[int]) -> str:
out, start, prev = [], None, None
for n in sorted(nums):
if start is None:
start = prev = n
elif n == prev + 1:
prev = n
else:
out.append(f"{start}" if start == prev else f"{start}-{prev}")
start = prev = n
if start is not None:
out.append(f"{start}" if start == prev else f"{start}-{prev}")
return ",".join(out)
def cmd_diff(args: argparse.Namespace) -> int:
cov = load(args.coverage)
with open(args.diff, encoding="utf-8") as fh:
added = parse_diff(fh.read())
total = covered = 0
unmatched = []
rows = []
for path, lines in sorted(added.items()):
fc = match(cov, path)
if fc is None:
unmatched.append(path)
continue
exe = [n for n in lines if n in fc.lines]
if not exe:
continue
miss = [n for n in exe if fc.lines[n] == 0]
partial = [n for n in exe if n in fc.branches and not all(fc.branches[n]) and fc.lines[n] > 0]
total += len(exe)
covered += len(exe) - len(miss)
rows.append((path, len(exe), len(miss), miss, partial))
print(f"{'file':60} {'changed':>7} {'missed':>6} uncovered lines / partial branches")
for path, n, m, miss, partial in sorted(rows, key=lambda r: -r[2]):
extra = f" partial: {ranges(partial)}" if partial else ""
print(f"{path[-60:]:60} {n:7d} {m:6d} {ranges(miss) or '-'}{extra}")
if unmatched:
print(f"\nnot in coverage report (unmeasured, not 'covered'): {', '.join(unmatched)}")
if total == 0:
print("\nno executable changed lines found in the coverage report")
return 0
pct = 100.0 * covered / total
print(f"\ndiff coverage: {covered}/{total} executable changed lines = {pct:.1f}%")
if args.fail_under is not None and pct < args.fail_under:
print(f"FAIL: below threshold {args.fail_under:.1f}%")
return 1
return 0
def cmd_worst(args: argparse.Namespace) -> int:
cov = load(args.coverage)
rows = []
t_lines = t_hit = t_br = t_br_hit = 0
for path, fc in cov.items():
n = len(fc.lines)
if n == 0:
continue
hit = sum(1 for h in fc.lines.values() if h > 0)
br = sum(len(v) for v in fc.branches.values())
br_hit = sum(sum(v) for v in fc.branches.values())
t_lines, t_hit, t_br, t_br_hit = t_lines + n, t_hit + hit, t_br + br, t_br_hit + br_hit
rows.append((path, n, n - hit, 100.0 * hit / n, (100.0 * br_hit / br) if br else None))
rows.sort(key=lambda r: (-r[2], r[3]))
print(f"{'file':60} {'lines':>6} {'missed':>6} {'line%':>6} {'branch%':>7}")
for path, n, miss, lp, bp in rows[: args.top]:
bps = f"{bp:6.1f}%" if bp is not None else " -"
print(f"{path[-60:]:60} {n:6d} {miss:6d} {lp:5.1f}% {bps}")
if t_lines:
bsum = f", branches {100.0 * t_br_hit / t_br:.1f}%" if t_br else ""
print(f"\ntotal: lines {100.0 * t_hit / t_lines:.1f}% ({t_hit}/{t_lines}){bsum}")
return 0
def main() -> int:
p = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
sub = p.add_subparsers(dest="cmd", required=True)
d = sub.add_parser("diff", help="coverage of lines added by a unified diff")
d.add_argument("coverage", help="lcov.info or Cobertura coverage.xml")
d.add_argument("diff", help="unified diff file (git diff -U0 base...HEAD)")
d.add_argument("--fail-under", type=float, default=None)
d.set_defaults(func=cmd_diff)
w = sub.add_parser("worst", help="files ranked by missed lines")
w.add_argument("coverage")
w.add_argument("--top", type=int, default=20)
w.set_defaults(func=cmd_worst)
args = p.parse_args()
try:
return args.func(args)
except (OSError, ET.ParseError, ValueError) as exc:
print(f"error: {exc}", file=sys.stderr)
return 2
if __name__ == "__main__":
sys.exit(main())
About this skill
Written and maintained by the Claude Skills editors to the same editorial standard as every skill in the directory: a selective trigger, a real workflow, known pitfalls and a verification step. First published , last updated . MIT licensed.