kb-linting

kb-linting

Periodic semantic health-check on a knowledge base. The mechanical part — schema integrity, edge resolution, orphan args, dangling-evidence args, unreferenced sources, status-stale candidates — is handled by scripts/kb-tool.py lint. This skill covers the part the script can't do: judgment calls about whether the graph still reflects the topic accurately, whether traceback has gone deep enough, whether the public discourse on the topic has shifted in ways the KB hasn't yet absorbed.

Run regularly (once per significant new ingest, between phases of construction, before kicking off a debate). Karpathy's three core operations are ingest / query / lint; this is the lint operation, model-driven.

Inputs

• A run directory with a populated kb/.
• The most recent mechanical lint report at <run-dir>/kb/.lint-report.json (run python3 <plugin>/scripts/kb-tool.py lint <run-dir> first if it doesn't exist).

If kb/ doesn't exist or the run-dir is empty, stop and tell the caller.

What to check (the semantic part)

For each check below: read the relevant slice of the KB, identify findings, decide acknowledge / fix immediately / queue for next research pass. Apply fixes via Write/Edit; the plugin's hooks re-index automatically.

1. Central-question coverage drift

kb/quality-summary.md declares the topic's central questions and which arg-IDs addresses each. Over time the graph shifts:

• Were any central questions identified that have zero addresses edges in the current graph? That's a coverage gap. Either find arguments that should engage them and add addresses edges, or queue gap-filling research.
• Did new arguments emerge that engage a central question without an addresses edge being recorded? Add the edge.
• Did the set of central questions shift? If new central questions emerged from the work but quality-summary.md wasn't updated, update it.

2. Engagement asymmetry

For each position represented in the KB, count how many of the central questions its arguments engage via addresses edges vs. how many it only relates to via tangential-to / scope-mismatch-with / layer-shift-from. Persistent imbalance is a finding worth noting in kb/log.md or kb/quality-summary.md — and it will already shape the debate skill's verdict downstream, so flagging it early is useful.

3. Traceback completeness

For each argument with basis_type: empirical or mixed:

• Has the traceback gone all the way to a primary source? (Re-read the Traceback notes section of the arg file. If it cites a meta-analysis or review, has the meta-analysis's primary studies themselves been examined?)
• If the traceback dead-ends, is the dead end recorded as a finding rather than implied?
• For arguments rated robust or strong — is the rating actually justified by what's in the corpus, or has it inherited a rating from when the corpus was thinner?

4. Contradiction-status drift

For each contradiction in kb/contradictions.md:

• Is its status (open / resolved-by-arg-NNN / unresolvable-given-current-corpus) still accurate? Sometimes a contradiction gets functionally resolved by later sourcing without anyone updating the status.
• Are the Involved arguments and Involved sources lists still complete? New arguments may have been added that bear on the contradiction.

5. Entity-page opportunities

kb/entities/ holds profiles for people, organizations, key studies, and named concepts that recur across the corpus. Use entity pages when consolidating scattered information helps:

• An author appears as the primary or coauthor of 3+ sources → consider an entity page (especially if their work concentrates a position's evidence base — e.g., a small group of co-authors generating most of the favorable primary studies)
• An organization (Cochrane, USPSTF, WHO, FDA, NICE, etc.) appears as the source / subject of 3+ sources or arguments
• A specific study is cited / discussed by 3+ arguments → entity page consolidating its methodology, criticism, replications
• A named concept / framework (informed consent, harm principle, network meta-analysis, the precautionary principle) is referenced across the graph → entity page with type: concept

For each candidate from the criteria above that doesn't have an entity page, decide: create one now, or note as a queue item.

6. Author-network blind spots

For each peer-reviewed source: has the Author network field been populated? In topics where author concentration is itself a finding (small clusters dominating the literature), missing networks are a real blind spot. Cross-reference with entity pages — if an entity page exists for the author, the network analysis lives there; if not and 3+ sources by the same author exist, create the entity page.

7. Source quality drift

For each source: re-read its Quality notes field. After more sourcing, has any source had its methodology effectively criticized by a newer source in the corpus? Is its Criticism field populated with cross-references? Has anything been retracted/corrected since access date that should move it to kb/sources/retracted/ or kb/sources/corrected/?

8. Stale-status verification

The mechanical lint surfaces status-stale-candidate findings for contested and weak arguments. For each:

• Is the rating still accurate? Sometimes an argument gets refuted in passing without anyone updating the status.
• If the rating is outdated, downgrade per the knowledge-base-construction skill's strength-rating discipline.
• If the rating is correct, acknowledge the finding and move on.

9. Mechanical findings — judge each

Walk the script's mechanical findings list and apply judgment:

• orphan-arg: deliberately new (just created, edges pending) or actually disconnected? If disconnected, find or create the missing edges; if no edges make sense, the argument may not belong in the graph.
• dangling-empirical-basis: does the argument actually rest on a source we've cached, but the derived-from-source edge wasn't added? Add the edge. Or did the source never get written? Then the argument's basis_type is wrong (set to reasoning or mixed).
• source-unreferenced: intentional background reading, or should an argument cite it?
• missing-engagement: the argument has outgoing edges but no addresses edge to another argument. Does it engage a central question that simply lacks the edge (add the addresses edge), or is it legitimately peripheral (acknowledge it, no change)?
• status-stale-candidate: see check #8.

Output

After working through the checks, append one synthesis entry to <run-dir>/kb/log.md:

## [YYYY-MM-DD HH:MM UTC] lint | <one-line summary>

- mechanical findings: <count> total — <count> acknowledged, <count> fixed, <count> queued
- semantic findings: <count> total — <count> acknowledged, <count> fixed, <count> queued
- key items addressed: <bullet list>
- queued for next research pass:
  - <bullet list with arg-ID / source-path references>

This single entry is the lint pass's narrative artifact. The mechanical report stays at kb/.lint-report.json (machine-managed by the script — do not write to it directly).

When to run

• After a significant new source-ingest pass during KB construction
• Before transitioning between construction phases
• Before kicking off /debate <run-dir>
• Periodically when the KB is being maintained over weeks (this is the Karpathy "wiki maintenance" pattern — lint is what keeps the graph honest as it grows)

What this skill does NOT do

• Does not mutate kb/.lint-report.json (script-owned)
• Does not write to indexes (hook-owned; Write/Edit triggers index rebuilds)
• Does not run mechanical schema validation (script-owned via kb-tool.py lint)
• Does not advocate or pick sides — same neutrality as the moderator agent
• Does not promote argument statuses (only KB construction can promote; lint can downgrade based on traceback findings, but if you find evidence that would promote something, queue it as a research-pass item rather than promoting in place)