research-investigation

Research Investigation

A scientific-rigor discipline for work whose deliverable is a defensible claim. The discipline IS the product: a finding nobody can reproduce or that skipped its controls is worth nothing, however exciting. Apply whenever you're testing hypotheses, analyzing data/artifacts, or writing an evidence-backed report.

Standing rules (non-negotiable)

• Pre-register before running. State the hypothesis, the method, and the decision rule (what result would confirm vs. falsify) before you see the output. A rule decided after the fact isn't a result.
• Same pipeline, same thresholds everywhere. Don't switch metrics or cutoffs between cases — that manufactures significance.
• Controls are mandatory. Every claim needs the right control:
  • Positive control / keystone — run the method on a known-answer case. If it can't find the signal where the signal is known to exist, every negative you report is uninterpretable. Report that prominently either way.
  • Negative / within-class control — compare like with like; never compare a privileged case to your subject and call the difference meaningful.
  • Shuffle / reversal for any sequential or directional claim.
• A falsified hypothesis is a result. Write "no" plainly. Negative and inconclusive results get the same care as positives — use a calibrated- negative format: show the method reached its floor on a known case (e.g. AUC ≈ chance), then state the upgrade path. "We couldn't detect X" ≠ "X is absent."
• Capability-bound caveats. Distinguish "X is false" from "this method can't see X." Close a door only as far as the evidence allows; never overclaim.
• Self-correct out loud. When a later check overturns an earlier finding, WITHDRAW it explicitly and propagate the correction everywhere it leaked. Verify notes against the primary — re-reading a source has reversed wrong "corrections."
• Honest frame. The standing claim is "replication + novel instruments + quantified disagreements," not "solved." Tag every claim NEW / EXTENSION / REPLICATION against the prior-art ledger; extensions cite the prior work.

Per-finding loop

1. Pre-register (hypothesis + method + decision rule).
2. Run on the control first (known-answer) — confirm the method works.
3. Run on the subject; record facts separately from interpretation.
4. Run the within-class / within-stratum control before writing anything down.
5. Save the evidence (crops/figures with regenerable coordinates; raw output).
6. Label the claim (NEW/EXT/REP, VERIFIED/REPORTED/UNVERIFIED) + a counter-reading.
7. Commit per completed unit with the one-line finding in the message.

Writing it up — read the primary, don't summarize

The biggest failure mode of a synthesis pass is writing from your own .md reading-notes without re-opening the source. Forbidden. Every source a claim leans on must be READ by the agent writing that claim, from the source itself — not its paraphrase. Gate completion with a source-engagement ledger: one row per section × the primaries it rests on, each marked READ / FETCHED / BROWSED / BLOCKED. A section with an unaccounted load-bearing primary is not finished.

Failure modes (hard-won)

• Don't trust orientation/axis/encoding conventions — verify with a synthetic known-answer test before interpreting any directional statistic.
• Don't let tooling artifacts (threshold/RANSAC/parser quirks) become "findings" — calibrate on a known case first.
• If a result is exciting, run the control BEFORE writing it down anywhere.

See PROVENANCE.md for sources.