populate-research-docs

Populate Research Docs

Mines the project's existing artifacts (git, README, code, notebooks, papers, prior docs/) to fill in the docs/ folder produced by the bootstrap-research skill. Designed to be re-run anytime — first-time onboarding, periodic refresh, or after a paper drop.

Core principle

Read, don't invent. Every line written must trace back to a concrete source in the repo. When the source is missing or ambiguous, leave the existing TODO: marker and report it as needing human input. A wrong fact is worse than a TODO:.

When to run

• After bootstrap-research (the natural pairing — bootstrap scaffolds, populate fills).
• When cloning a research repo and wanting a fast docs/ summary of what's there.
• Periodically (every ~5–10 experiments, or end of a research thread) to refresh findings.md from experiments.md.
• After dropping new PDFs into papers/ or appending to .bib — to refresh literature.md.
• When the user says: "populate docs", "fill in the TODOs", "refresh findings", "update literature from papers/", "summarize this repo into CLAUDE.md".

Preconditions — bail out if not met

Before any work:

1. Run pwd and ls -la. Confirm cwd looks like a real project root (not $HOME / mono-repo root). If unclear, ask which directory to work on.
2. Check CLAUDE.md and docs/ exist. If neither exists → tell the user to run bootstrap-research first; do not silently create them. If only CLAUDE.md exists (no docs/), ask whether to scaffold docs/ first via bootstrap, or proceed with CLAUDE.md-only updates.
3. Check git is clean OR warn the user before writing — populate makes multi-file edits and a clean tree makes it easy to review/revert.

Sources to mine

Run these in parallel where possible:

Source	Tool	Feeds
git log --oneline -50	bash	progress.md recent activity, contributor list
git log --since='3 months ago' --pretty=format:'%h %ad %s' --date=short	bash	recent decisions / progress
README.md	Read	overview.md (research question, scope, methodology)
pyproject.toml / requirements.txt / Cargo.toml	Read	overview.md tech stack section, CLAUDE.md tech stack
*.ipynb (top-level + notebooks/)	bash + jq/python	experiments.md candidate entries, findings.md if results explicit
*.bib, references.bib	Read	literature.md Read / Reading queue
papers/, references/, lit/	ls	literature.md reading queue (PDF filenames)
experiments/, runs/, logs/	ls -R	experiments.md ID inference, architecture.md runs layout
src/, *.py outside notebooks	bash + grep	architecture.md swap points and pipeline shape
Existing docs/experiments.md	Read	source for findings.md distillation
Existing docs/decisions.md	Read	cross-check against commit messages mentioning "decided"

Per-file population rules

For each docs file: read first, then merge conservatively. Never overwrite a non-TODO line. Insert auto-content under a clearly marked block:

<!-- auto-populated YYYY-MM-DD by populate-research-docs — review and edit -->
<auto-content here>
<!-- /auto-populated -->

Re-running the skill replaces only the content inside these markers (idempotency). User edits outside the markers are preserved.

overview.md

• Research question: only fill if README has a clearly-phrased question (ends with ?, or has a "Research question:" / "We ask:" / "Hypothesis:" line). Otherwise leave the TODO and report it.
• One-line description: condense README's first paragraph to one sentence. Skip if README is missing or just a title.
• Scope: leave TODO unless README has explicit "in scope" / "out of scope" / "we do not address" language.
• Methodology summary: list data sources and key libs detected; explicit metric/baseline only if found in code or README.

progress.md

Append a new dated section (today) summarizing recent git activity:

## YYYY-MM-DD — auto-populated snapshot

### Recent activity (last 30 days)
- <commit subject> (<hash>, <date>)
- ...

### Detected state
- Notebooks: N total, last modified <date>
- Experiment outputs: <runs/ subfolders count, latest>
- Open TODOs in code: <grep count>

Do not rewrite past progress.md entries. Append only.

experiments.md

Highest-risk file. Be conservative.

• If existing entries are present: do not modify them.
• If experiments/ or runs/ directories have subfolders that look like experiment outputs (config + log + checkpoint pattern), draft stub entries under a section titled ## Detected runs (auto, please verify):

  ## YYYY-MM-DD-auto — <subfolder name>
  **Setup** (auto-detected):
    - Code: <script or commit if findable>
    - Params: <from config.yaml/json if present>
    - Seed: <if logged>
  **Result**: TODO: pull from <log path>
  **Hypothesis**: TODO: human input — not inferable
  

  Never invent the hypothesis. Leave it TODO: so the user knows to fill it.
• If notebooks contain a markdown cell that explicitly states a hypothesis or result, suggest an entry but mark it [UNVERIFIED] and reference the notebook + cell number.

findings.md

Only run distillation if experiments.md has ≥3 real (non-stub) entries. Otherwise skip and report.

• Scan completed entries' "Observations" and "Result" fields.
• Group into Confirmed / Disconfirmed / Open / Surprises buckets only when the source entry uses clear language (works, does not improve, unclear, surprising).
• Every line in findings.md must reference the experiment ID it came from.
• If the user already wrote findings, append to the appropriate bucket inside the auto-populated block; do not deduplicate or reword user lines.

literature.md

• Parse .bib files: extract @article / @inproceedings entries → put in Read if the user has cited them anywhere (grep for the bibkey across project), else Reading queue.
• List PDFs in papers/ / references/ / lit/ → Reading queue with the filename. Do not invent takeaways.
• Cross-reference: if a paper is mentioned in README or comments, note where ("cited in src/loss.py:42").

decisions.md

• Scan git log for commit messages matching decided|decision|chose|settled|switched to|going with → suggest an entry per match with the commit date and message.
• Mark all auto-suggested entries with [AUTO — verify] prefix so the user can confirm and clean up.
• Do not scrape decisions from code comments unless they say "decision:" explicitly.

architecture.md

Only update if file exists (bootstrap creates it conditionally). Do not create it if missing — defer to bootstrap's signal logic.

• Pipeline shape: infer from imports + script structure. If src/data.py → src/model.py → src/train.py exists, draft an ASCII flow. Mark as auto-detected.
• Swap points: grep for config-driven instantiation patterns (hydra.utils.instantiate, getattr(models, ...), MODEL_REGISTRY). List what was found.
• Frozen interfaces: do not auto-fill — this requires human judgment about which contracts are load-bearing.
• Repo layout: top-level dir tree with one-line descriptions, only for dirs whose purpose is obvious from name or contents.

todo.md

Scan codebase for TODO: / FIXME: / XXX: comments. Append a section:

## Detected in code (auto, YYYY-MM-DD)
- [ ] `path/to/file.py:42` — <comment text>

Group by directory if list exceeds 20 items. Don't touch the user's Now / Soon / Later sections.

CLAUDE.md

Touch only if there are unresolved {{PLACEHOLDER}} markers or TODO: in the Tech stack / File map / Hard rules sections — fill from current project state. Do not change the workflow / conventions sections (those are user-editable templates, not auto-content).

Hard don'ts

• Don't fabricate research questions, hypotheses, results, or confirmed findings. TODO: is always the safe choice.
• Don't overwrite content outside the <!-- auto-populated --> markers. User edits are sacred.
• Don't run if CLAUDE.md and docs/ are absent — tell the user to bootstrap first.
• Don't bloat any file past readable size. If auto-content would exceed ~80 lines in one file, summarize and put detail in a sibling file (e.g., literature_full.md).
• Don't echo this skill's contents to the user.
• Don't make experiments.md entries look real if they're auto-stubs — always mark auto / UNVERIFIED / [AUTO — verify] so the user can spot and clean them.

Final report

After running, print a concise summary:

• Per-file diff: overview.md: filled 2 of 4 TODOs, findings.md: 0 entries (need ≥3 experiments first), etc.
• List of remaining TODO: markers that need human input — this is the user's punch list.
• One-line reminder: "Re-run anytime; auto-content is bounded by <!-- auto-populated --> markers and won't clobber your edits."