Reproducible Research skill for Claude Code

Done well means: a stranger with the repo, the data access, and one documented command regenerates every figure and number in the paper, from raw data, with the same results (bit-identical where feasible, within stated tolerance otherwise), and the exact code, environment, and data versions are archived with persistent identifiers.

Hard rules

• Raw data is read-only. Never edit raw files in place; every transformation is code.
• Every number and figure in the manuscript is produced by a script in the pipeline, not copied by hand from a notebook session.
• Pin everything: language version, packages (lock file), system libraries (container or documented), data version (hash), and random seeds.
• Never commit secrets, credentials, or personal data. Restricted data stays out of git and out of public archives; document how to obtain it.
• Before deleting cached data or remote storage (dvc gc, bucket cleanup), show what will be removed and get user confirmation.

Reproducibility levels

Level	What a reader can do	Minimum setup
1. Shared	Read the code	Public repo with license
2. Re-runnable	Run it with effort	README with steps, dependency list
3. Reproducible	Regenerate results with one command	Lock file, data versioning, pipeline, seeds
4. Durable	Still works in 5+ years	Container image archived, code+data archived with DOIs

Target level 3 for any paper; level 4 for flagship results and theses.

Repository layout

project/
├── README.md              # question, how to reproduce (one command), data access
├── data/
│   ├── raw/               # immutable inputs (DVC-tracked or downloaded by script)
│   ├── interim/           # intermediate, regenerable
│   └── processed/         # analysis-ready, regenerable
├── src/                   # importable functions (tested)
├── pipeline/ or dvc.yaml  # stages from raw to outputs
├── notebooks/             # exploration; numbered; not the source of final results
├── results/{figures,tables,numbers}/
├── paper/                 # manuscript pulls from results/
├── pyproject.toml + uv.lock   (or environment.yml + conda-lock.yml, renv.lock)
├── params.yaml            # all tunable parameters and seeds
└── CITATION.cff, LICENSE

Workflow

- [ ] 1. Environment: pin and lock
- [ ] 2. Data: version raw inputs; document provenance and license
- [ ] 3. Pipeline: declare stages raw -> outputs
- [ ] 4. Determinism: seeds, threads, sorted iteration
- [ ] 5. Notebooks: clean execution or convert to scripts
- [ ] 6. Manuscript numbers generated, not typed
- [ ] 7. Clean-room rerun
- [ ] 8. Archive and cite (Zenodo DOI, availability statement)

1. Environment

Stack	Default	Commands
Python	uv with pyproject.toml + uv.lock	uv init, uv add pandas==2.*, uv lock, uv sync --frozen, uv run python ...
Python + compiled/GPU/non-Python deps	conda/mamba + conda-lock	conda-lock -f environment.yml -p linux-64 -p osx-arm64; conda-lock install -n proj conda-lock.yml
R	renv	renv::init(), renv::snapshot(), renv::restore()
Anything, long-term	Container	Dockerfile from a digest-pinned base image; push the image and record its digest
Fully hermetic	Nix flakes	See nix-environments

Record the Python/R version (requires-python, .python-version, R version in renv.lock). For GPU work, record CUDA/cuDNN and driver versions.

2. Data versioning with DVC

uv add dvc dvc-s3              # or dvc-gs, dvc-azure, dvc-ssh; DVC 3.x
dvc init
dvc remote add -d storage s3://my-bucket/project-dvc
dvc add data/raw/survey_2025.csv      # creates .dvc pointer file, adds data to .gitignore
git add data/raw/survey_2025.csv.dvc data/raw/.gitignore .dvc/config
git commit -m "Track raw survey data with DVC"
dvc push                              # upload content-addressed data to the remote
# collaborator
git clone ... && dvc pull

Remote credentials live in .dvc/config.local (dvc remote modify --local ...) or environment/cloud credential chains, never in the committed .dvc/config.

Document provenance for each raw input in data/README.md: source URL or DOI, access date, license/terms, checksum, and any access restrictions. For public downloads, fetch with a script and verify a SHA-256 instead of committing the file.

Alternatives: git-annex or DataLad for very large or many-file datasets; Git LFS for a few medium binary files (no pipeline features).

3. Pipeline

dvc.yaml declares stages with dependencies, parameters, and outputs; dvc repro reruns only what changed.

stages:
  clean:
    cmd: uv run python src/clean.py data/raw/survey_2025.csv data/processed/clean.parquet
    deps: [src/clean.py, data/raw/survey_2025.csv]
    params: [clean.min_age]
    outs: [data/processed/clean.parquet]
  model:
    cmd: uv run python src/model.py
    deps: [src/model.py, data/processed/clean.parquet]
    params: [model.seed, model.n_estimators]
    outs: [results/models/model.pkl]
    metrics: [results/numbers/metrics.json: {cache: false}]
    plots: [results/figures/roc.csv: {x: fpr, y: tpr}]

dvc repro                 # run what is stale
dvc dag                   # show the graph
dvc status                # what changed vs. dvc.lock
dvc params diff; dvc metrics diff main   # compare against another revision
dvc exp run -S model.n_estimators=500    # experiment without committing

Commit dvc.lock; it pins the hash of every input and output.

Other good choices: Snakemake (bioinformatics, HPC clusters), R targets (R projects), Make (small projects; see task-runners). Pick one; do not mix.

4. Determinism

import os, random
import numpy as np
SEED = params["model"]["seed"]
random.seed(SEED); np.random.seed(SEED)
rng = np.random.default_rng(SEED)          # prefer passing rng explicitly
os.environ["PYTHONHASHSEED"] = str(SEED)   # only affects subprocesses started after this
# PyTorch
import torch
torch.manual_seed(SEED); torch.use_deterministic_algorithms(True)
# also set CUBLAS_WORKSPACE_CONFIG=:4096:8 in the environment for CUDA determinism

• Sort before iterating over sets, dict-from-filesystem listings, and glob results.
• Multithreaded BLAS and GPU kernels can produce tiny floating-point differences; state the tolerance (e.g., metrics equal to 1e-6) when bit-identical results are not feasible.
• Record timestamps and “today” as parameters; never call now() inside analysis logic.

5. Notebooks

• Exploration only, or parameterized execution with papermill/Quarto as a pipeline stage.
• Before committing: restart kernel and run all; strip outputs (nbstripout --install) or pair with a text format (jupytext) so diffs are reviewable.
• Move reusable code into src/ with tests. See jupyter-notebooks for notebook hygiene.

6. Numbers in the manuscript

Write every reported number to results/numbers/*.json and pull them into the paper (LaTeX macros generated by a script, Quarto inline code, or knitr). Example generator output: \newcommand{\nParticipants}{1{,}204}. A typo-free manuscript is a side effect.

7. Clean-room rerun

git clone <repo> /tmp/rerun && cd /tmp/rerun
uv sync --frozen && dvc pull
dvc repro --force          # or: snakemake --cores 4 --forceall
git status --porcelain     # outputs tracked by dvc.lock should be unchanged
dvc status                 # "Data and pipelines are up to date."

Better: run it in CI or a fresh container on a different machine. Compare key numbers against the manuscript automatically.

8. Archive and cite

• Tag the release (v1.0-paper), archive it on Zenodo via the GitHub integration to get a DOI; add CITATION.cff.
• Archive data on a repository with a DOI (Zenodo, Dryad, OSF, Figshare, or a domain repository); restricted data: deposit metadata and access procedure.
• Availability statement: code DOI, data DOI or access route, environment (lock file / image digest), and the one command to reproduce.

Gotchas

• pip freeze captures one platform; use a cross-platform lock (uv, conda-lock).
• DVC outputs listed in outs are deleted and regenerated by dvc repro; do not hand-edit them.
• A notebook that only runs in the author’s cell order is not reproducible; “Run all” in a fresh kernel is the test.
• Container latest tags drift; pin base images by digest.
• Licensed or personal data in a public archive cannot be fully unpublished. Check terms before depositing.