litmus

Litmus

Reality-check an AI-generated document before you implement it.

Litmus is a Claude Code plugin. Given an architecture doc, RFC, spec, or plan, it runs a five-phase pipeline that breaks frame-anchoring by structural force: it atomizes the doc's claims, re-derives the design from the problem statement alone (without seeing the source), grounds every load-bearing claim against the web via Claude's inbuilt exa, attacks the doc through five orthogonal critic lenses, scores the critics for sycophancy, and emits an evidence-first report.

There is no verdict line. Humans own the call.

/litmus path/to/architecture.md

Why this exists

A team I know was handed an architecture document by a vendor and an engineer implemented it faithfully. Afterward it turned out the architecture was AI-generated and the proposed system didn't make sense. Every AI they asked to evaluate the doc said "looks correct", because it was checking internal consistency, not whether the doc described a system that should exist.

That failure has a name: frame-anchoring. LLMs treat their prompt as the world. Sycophancy compounds it. Asking "is this correct?" gets you agreement, not truth.

The leading prior-art tools (EveryInc's compound-engineering plugin, PlanExe's premise-attack ensemble, arch-review-assistant's 9-agent panel) all do clever adversarial review within the doc's frame. None ground claims against external reality. That is the gap Litmus closes.

See docs/why-litmus-is-different.md for the full comparison.

Install

Litmus is pre-publish. Install from source while it's still early.

Prerequisite. Litmus requires Claude's inbuilt exa (mcp__claude_ai_Exa_2__*), which ships with Claude.ai sessions. The grounder aborts the pipeline with an actionable message if those tools are not reachable.

Litmus does NOT use a locally-installed exa MCP server. If you have one configured (mcp__exa__*), Litmus will not call it: that path bills against your personal API key, which the plugin treats as out-of-scope without explicit per-run permission.

Install Litmus from this repo:

claude plugin marketplace add itzTiru/litmus
claude plugin install litmus@litmus-marketplace

For local iteration without installing:

git clone https://github.com/itzTiru/litmus.git
cd litmus
claude --plugin-dir ./plugins/litmus

Then run /litmus examples/bad-architecture.md in the resulting Claude Code session.

Use it

/litmus path/to/your/doc.md

Output lands in ./litmus-reports/<YYYYMMDD-HHMMSS>/:

• report.md: primary deliverable.
• report.html: same content, viewable in any browser. No external assets.
• atoms.json: Phase 0 atomization.
• independent-design.md: Phase 1 fresh design, produced by a subagent that never saw the source doc.
• citations.json: Phase 2 grounding results.
• lenses/<name>.json: one file per activated Phase 3 lens.
• sycophancy.json: Phase 4 collapse scores.
• audit/: pre- and post-re-prompt lens outputs, kept for traceability.

Open report.html in a browser. The "Independent Re-derivation Diff" section is the one to read first. It surfaces where the doc made choices a fresh designer would not.

What it does

The five phases, in order:

1. Atomize. Extracts every load-bearing claim as an atomic proposition tagged problem, solution, assumption, prediction, or numeric. Separates what the doc says from how it argues.
2. Re-derive. Spawns a subagent that receives only the problem-tagged atoms. The source doc is intentionally withheld from its prompt. It produces a fresh design with three ranked alternatives (including a do-nothing baseline).
3. Ground. For each load-bearing claim, searches the web via Claude's inbuilt exa (mcp__claude_ai_Exa_2__*) and assigns one of GROUNDED, CONTRADICTED, UNGROUNDED, UNFALSIFIABLE. UNGROUNDED is a finding, not a passing grade.
4. Attack. Five orthogonal critic lenses run in parallel (Integrity, Accountability, Spectrum, Cascade, Escalation). Each declares the other four's territories as out-of-scope. Each finding uses a discrete confidence anchor (0, 50, 75, 100) tied to a behavioral criterion the lens must self-apply.
5. Score sycophancy and synthesize. A judge scores each lens on five signals (template language, hedging without specifics, agreement without evidence, collaborative shape, anchor inflation). Collapsed lenses get re-prompted. A synthesizer reads all the artifacts and emits the markdown report; a bundled Python script (stdlib only) renders it to HTML.

A deeper walk-through is in docs/how-it-works.md.

What it doesn't do