llm-externalizer

Assess ONE OpenRouter model against EVERY LLM tool's per-tool REQUIREMENTS (TRDD-f45eeaa0) — free: makes NO LLM call (no token cost), only a public model-catalog fetch (no API key). Reports, per tool, whether the model meets that tool's hard requirements (cost / context / output / params) and which of the qualifying tools ALSO have a benchmark gate to run before assignment. Does NOT run any benchmark. Use to vet a candidate model across the whole tool surface at a glance. Trigger with "assess model X", "which tools can model X serve", "does model X meet the requirements".

Check, for every LLM tool that has a per-tool benchmark (security_scan, search_existing_implementations), whether its configured model has DEGRADED (per the durable model-health ledger) and, when so, run that tool's ADVISORY benchmark to surface the best SAME-OR-CHEAPER replacement. READ-ONLY by default — it writes a report and recommends, it never rewrites your settings. A pricier model is NEVER recommended. Trigger with "check tool replacements", "is any tool model degraded", "audit my per-tool models", or "auto-replace my LLM tool models". To actually adopt a recommendation, run the CLI `--apply` (below).

Benchmark every model in the active profile's free pool (or the bundled FREE_POOL_SEED if none is pinned) with one invocation. Same scoring as /llm-externalizer-benchmark but auto-fills the candidate set from the free-model list and refuses to run if any entry is not a ':free' model. Use this after switching free_only on, or to evaluate which free model best replaces a paid model.

Benchmark OpenRouter programming-category models against a TypeScript classification task. Filters by cost + capability, scores each candidate against 71 fixture functions + 3 literal keywords, writes a markdown comparison report. Use this to pick the cheapest model that still passes the real workload.

Redirect to user-only manual model/ensemble configuration. There is NO change_model MCP tool — the server is read-only by design. This command explains how to edit ~/.llm-externalizer/settings.yaml by hand and reload with reset, and points you at the optional benchmark / assess-model helpers for picking which models to use.

Opus-model variant. Verify and fix ONE LLM Externalizer per-file bug report. Input is a single absolute path to a report `.md`. Validates findings, applies minimal fixes only to REAL bugs, runs linters, writes a `.fixer.`-tagged summary, returns the summary path. Dispatched in parallel by `llm-externalizer-scan-and-fix` when the user picks "opus" on the model-menu prompt.

Sonnet-model variant. Verify and fix ONE LLM Externalizer per-file bug report. Input is a single absolute path to a report `.md`. Validates findings, applies minimal fixes only to REAL bugs, runs linters, writes a `.fixer.`-tagged summary, returns the summary path. Dispatched in parallel by `llm-externalizer-scan-and-fix` when the user picks "sonnet" on the model-menu prompt.

Use for a fast code review from the LLM Externalizer ensemble without loading scan output into the main context. Accepts a file/folder/glob and returns only report paths. Trigger with "review this file", "llm-ext review", "audit these files", "scan for bugs".

Opus-model variant. Fix exactly ONE bug from a markdown bug list produced by llm-externalizer-fix-found-bugs. Reads the bug-file absolute path, picks the highest-severity unfixed entry, applies a minimal surgical fix, updates the bug file with a ` — FIXED` marker plus a short post-mortem, returns a single-line summary. Dispatched per-bug when the user picks "opus" on the model-menu prompt.

Sonnet-model variant. Fix exactly ONE bug from a markdown bug list produced by llm-externalizer-fix-found-bugs. Reads the bug-file absolute path, picks the highest-severity unfixed entry, applies a minimal surgical fix, updates the bug file with a ` — FIXED` marker plus a short post-mortem, returns a single-line summary. Dispatched per-bug when the user picks "sonnet" on the model-menu prompt.

Internal maintainer dogfood harness for the llm-externalizer plugin — exercises every surface (CLI verbs, benchmark, MCP-tool wiring, all slash commands, all skills) with a zero-spend offline sweep plus an opt-in free-pool live smoke. NOT user-invocable; run via tests/dogfood/dogfood_test.py. Use when validating a release or after touching any tool, command, or skill.

Hugging Face Hub CLI (`hf`) reference for downloading, uploading, and managing models, datasets, and repos. Covers custom --local-dir placement, --include/--exclude file filters, --revision pinning, cache management, and `hf auth login` for gated repos. Use when the setup wizard's pre-built download_command needs extension. Loaded by llm-externalizer-setup-agent.

Find the highest-scoring models for a coding task by querying the official Hugging Face benchmark leaderboards, with memory-budget filtering and per-device fit. Use when the recommender script returns no compatible row and the user has explicitly widened the search. Loaded by llm-externalizer-setup-agent.

Run rigorous evaluations against Hugging Face Hub models using inspect-ai or lighteval on local hardware. Covers backend selection (vLLM / Transformers / accelerate), local GPU evals, smoke tests, and task selection. Use when the user wants a deeper benchmark than the wizard's 5-test compatibility check. Loaded by llm-externalizer-setup-agent.

Select GGUF artifacts and quantizations for llama.cpp on CPU, Mac Metal, CUDA, or ROCm runtimes. Covers Q4_K_M vs Q5_K_M vs Q6_K trade-offs, llama-server launch flags, --hf-repo/--hf-file fallback for non-standard naming, and conversion from Transformers weights when no GGUF exists. Use when the user picks llama.cpp / LM Studio / Ollama on non-Apple-Silicon platforms. Loaded by llm-externalizer-setup-agent.