bot-battle-referee

Status: v0.1 — minimum viable. Full skill (config-driven sweeps) lands in Phase 3.

When to invoke

Trigger files: balance.ts, hard_vs_hard.ts, simulate.ts, any script invoking run_lobby / run_tournament. Trigger words: "run a balance check", "H2H tournament", "evaluate the new bot", "compare strategies".

The Rule

Before any bot run that will produce a balance/strategy claim:

1. Lobby composition is smart-only. All seats use the same difficulty (smart). Mixed lobbies are NEVER valid for balance claims (per global CLAUDE.md).
2. n ≥ 1000 per cell for balance claims (per global CLAUDE.md: "verify at n=1000 with CIs"). For exploratory runs, n=100 is fine but say so.
3. Sweep all relevant axes. Player count, founder ability, starting config — do not generalize from one cell.
4. Baseline is named. "Above baseline" means "above run dated at n=".
5. Seed handling is stated. Fixed seed for reproducibility OR random seed for population estimate — name which.
6. Test a 10-game loop first before launching the long run (per global CLAUDE.md).

Minimum viable checklist (v0.1)

1. Identify which script the user wants to run. If multiple candidates, ask.
2. Verify the script's default config matches the gate above (smart-only? n? sweep?).
3. If config violates the gate, surface it and propose a corrected config before running.
4. Run the 10-game loop first; confirm no crashes; then run the full n.
5. After completion, hand off results to bg-superpowers:balance-report-critic for the report gate.

Anti-patterns

• Running mixed-difficulty lobbies and reporting "balance is fine" — invalid signal.
• Reporting CMA-ES training WRs as final numbers — per global CLAUDE.md, these are inflated; require n=1000 verification.
• Running at n=100 and claiming a 2pp difference (CI swallows the signal).
• One-cell run ("3-player only") generalized to "balance is good".

Cross-references

• bg-superpowers:balance-report-critic is the report-side gate; this skill is the run-side gate.
• After a run, invoke bg-superpowers:anomaly-scanner on the produced game logs.
• superpowers:test-driven-development for new bot logic — failing seed first, then code.

TODO (Phase 3 full implementation)

• Per-project run config (bg-superpowers.config.json declares lobby defaults, sweep axes, baseline run id).
• Auto-launch the 10-game loop pre-flight check.
• Detect mixed lobbies in arg parsing and refuse the run.