multi-llm-review

Multi-LLM Review — Orchestrated Skill Audit

This skill turns the multi-LLM review workflow demonstrated on 2026-05-22 into a reproducible procedure. Use it as the semantic counterpart to skill-audit (which catches mechanical issues only).

When to use

• Before merging a new plugin to the marketplace — get cross-model perspective on utility and gaps.
• After substantive changes to a plugin (new skill, restructured workflow, new tool dependency).
• Periodically (quarterly?) on production-critical plugins to catch institutional drift.
• After running skill-audit and fixing P0 mechanical issues — the multi-LLM review will catch semantic issues (logic bugs, citation drift, missing prerequisites that require domain knowledge).
• NOT for every PR — this workflow costs ~5-15 min wall time and ~$1-3 in Codex API per plugin. Use judgment.

Workflow

The review runs in 4 phases. Phase A (FGCZ context build) has been absorbed into the fgcz-context skill, so it's a load-step rather than a phase.

Phase 0: load fgcz-context skill (already a skill, single load)
Phase C: per-plugin reviews in parallel (3 reviewers × N plugins)
Phase D: aggregate findings with agreement strength

Procedure

1. Detect available reviewers

bash meta-skills/skills/multi-llm-review/scripts/detect_reviewers.sh

Outputs one reviewer name per line. Stderr carries diagnostic notes for partial availability.

Three usable reviewers (claude + codex + cursor) → full mode. Fewer than three → solo mode (see references/solo_mode.md). The orchestrator should print the warning from references/solo_mode.md and confirm with the user before proceeding.

2. Render the per-plugin prompt for each target plugin

The prompt template is assets/per_plugin_review_prompt.md. Substitute:

• {{FGCZ_CONTEXT_PATH}} — typically meta-skills/skills/fgcz-context/SKILL.md
• {{PLUGIN_NAME}} — e.g. bfabric-lims
• {{PLUGIN_PATH}} — absolute path to plugin

3. Launch reviewers in parallel

For each available reviewer, launch in background. Save output to reviews/<YYYY-MM-DD>_<plugin>/<reviewer>.md.

Codex (GPT-5.5):

cat <prompt> | codex exec -m gpt-5.5 --dangerously-bypass-approvals-and-sandbox - \
  > reviews/<date>_<plugin>/codex.md 2> reviews/<date>_<plugin>/codex.err &

Cursor (Composer 2.5):

cat <prompt> | agent -p --model composer-2.5 --force --output-format text \
  > reviews/<date>_<plugin>/cursor.md 2> reviews/<date>_<plugin>/cursor.err &

Claude (Agent persona via Claude Code's Agent tool): Spawn a sub-agent that reads the prompt and writes the review to the same reviews/<date>_<plugin>/agent-persona.md path. Use subagent_type: general-purpose.

In solo mode, instead launch 3 Agent personas with the prompts in references/solo_mode.md (Security/Infrastructure, Usability/Documentation, Cross-skill Consistency).

4. Wait for completion

All reviewers run independently. Aggregate after the last one finishes (use wait for background bash jobs, or rely on Claude Code's task-completion notifications for Agent sub-agents).

Expect ~3-10 minutes per plugin for the slowest reviewer to finish.

5. Aggregate findings

Read all reviews/<date>_<plugin>/<reviewer>.md files. Produce an aggregate following assets/aggregation_template.md:

• Consensus utility verdict (HIGH / MEDIUM / LOW)
• Per-skill consolidated assessment (median rating, modal verdict)
• P0 findings ordered by agreement strength (4/4 first, then 3/4, etc.)
• Cross-plugin observations (if reviewing multiple plugins)
• Reviewer behavior notes (which reviewer caught what — calibrates future runs)
• FGCZ context validity check (did any reviewer find a conflict with fgcz-context content?)

Save the aggregate to reviews/<date>_<plugin>/_aggregate.md.

6. Update audit history

Append a one-line entry to reviews/index.md with:

• Date
• Plugin (or "marketplace" for full sweep)
• Reviewers used
• Top P0 count
• Link to aggregate file

This is consumed by skill-review-history (planned skill) for trend analysis.

Scope variants

Scope	Invocation	Cost (estimate)
Single skill	render prompt for the parent plugin but instruct reviewers to focus on one skill	~2 min, ~$0.10
Single plugin	one render per plugin	~5-10 min, ~$0.30
Full marketplace	N plugins × M reviewers in parallel (limit to ~12 parallel jobs)	~30 min, ~$2

Solo mode (concise)

If detect_reviewers returns only claude:

1. Print the warning from references/solo_mode.md.
2. Confirm with user.
3. Spawn 3 Claude Agent sub-agents with personas (Security/Infra, Usability, Cross-skill).
4. Aggregate as normal but mark the report as solo-mode in the header — same-model agreement is weaker evidence than cross-model agreement.
5. Recommend re-running with at least one external reviewer before committing fixes for any P0 finding.

What this skill does NOT do

• Auto-commit fixes. The aggregate is advisory. The user (or a separate "apply-fixes" workflow) decides what to land.
• Replace skill-audit. Run skill-audit first for mechanical issues. multi-llm-review is for the semantic complement.
• Replace human judgment. Reviewer LLMs have biases (Codex legalistic-strict, Cursor cross-skill-focused, Claude personas inherit base-model biases). 4/4 agreement is strong signal but never absolute.

See also

• meta-skills/fgcz-context — institutional preamble loaded by all reviewers
• meta-skills/skill-audit — deterministic linter (run first, cheap)
• assets/per_plugin_review_prompt.md — the Phase C prompt template
• assets/aggregation_template.md — the Phase D output structure
• references/tool_detection.md — CLI install / auth / probe details
• references/solo_mode.md — single-LLM fallback design and limitations
• references/output_format.md — file layout, severity policy, finding classification