Stats
Actions
Available In
Tags
Plugin
understudy
Improve LLM apps and agents from real traces — reduce cost/latency, raise quality/reliability, capture traces, build evals, run local optimization (GEPA), compare models/providers, and route through Understudy.
What's Inside
Skills22
distill-classifier
/distill-classifier
Use when a developer wants to replace an expensive frontier model on a classification workload (binary, multi-class, multi-label, or structured extraction) with a fine-tuned open-weight student — "distill this classifier", "can a small model do this tagging job", "the frontier labels these for $X, make it cheaper", "consensus-label my data". Multi-teacher majority-vote labeling, failure-directed SFT data, and a four-way promote/shadow/collect/stop verdict.
ingest-traces
/ingest-traces
Use when a developer already has production LLM traces — a bucket of captures, provider log exports, or gateway capture files — and wants them turned into local, redacted eval sets, or profiled for cost first. "Ingest my traces", "turn these logs into an eval set", "where is my LLM spend going", "which calls could a local model take over".
install-plugin
/install-plugin
Use when a developer wants to install, enable, update, reinstall, or verify the Understudy skills as a Claude Code plugin — "install understudy", "add the understudy skills", "set up the plugin", "why can't you see the understudy skill". Runs the non-interactive `claude plugin` CLI (or shows the commands), then tells the developer the one activation step and whether a restart is needed.
design-simulated-environment
/design-simulated-environment
Use to build a simulated, seeded environment (AutomationBench / verifiers style) so any model can run a captured agentic workload end-to-end and be scored on final state — "simulate this workload's tools", "build a validator for these traces", "let a small model attempt the whole task", "score recall/precision against gold", or any handoff from understand-workload toward whole-case model comparison.
compare-trajectories
/compare-trajectories
Use when you need to know HOW two model runs differ behaviorally on the same tasks, not just THAT one scores higher — per-task trajectory diffing that classifies the gap as persistence/recovery, knowledge, or format/parsing. "why does the bigger model pass these", "is this gap RL-shaped", "diff these two trajectory runs", "where do the trajectories diverge", "what would distillation buy me". The behavioral complement to compare-model-sweep.
Stats
Version0.3.0
LanguageTypeScript
Stars2
MaintenanceExcellent
LicenseMIT
Last Commit
Added
Available In
Tags
Categories
Keywords
