turing

turing

The research assistant that can't fool itself.

A Claude Code plugin that runs autonomous ML experiment loops, named after the man who first asked whether machines could think. Two agents enforce a strict separation: one writes code, one scores it, and neither can see the other's work. Immutable evaluation, anti-cheating guardrails, and structured hypothesis tracking make sure the results stay honest. When code is free, research is all that matters. You bring the research taste; Turing handles the rest.

• Separation: the agent modifies train.py; it cannot see or touch evaluate.py
• Memory: every hypothesis registered, every experiment logged, every variant preserved
• Convergence: automatic detection of diminishing returns; the agent stops when it should
• Taste: you inject ideas with /turing:try, read results with /turing:brief

[!NOTE] Turing is in active development. Some features are rough around the edges. Issues and feedback welcome.

Install

npm install -g claude-turing && claude-turing install --global && claude-turing verify

The Taste-Leverage Loop

You have taste: the accumulated judgment about which problems are tractable, which metrics matter, and which directions are dead ends. Turing has leverage: the discipline to run experiments without fatigue, track every result without amnesia, and measure without contamination.

The interface is two verbs:

/turing:try switch to LightGBM        Your taste → the agent
/turing:brief --deep                   The agent's results → you

Everything in between (experiment logging, convergence detection, hypothesis tracking, statistical validation, anti-cheating guardrails) is infrastructure connecting those two endpoints. You think about what to try. Turing handles how to try it.

What a Session Looks Like

/turing:init                          Scaffold a new ML project
/turing:train                         Agent runs 5-10 experiments autonomously
/turing:brief                         Campaign summary: what improved, what's exhausted
/turing:try "add polynomial features" Inject your next idea
/turing:train                         Agent follows your lead

For fully hands-off operation:

/loop 5m /turing:train

The agent trains, evaluates, keeps improvements, discards regressions, detects convergence, and stops. You come back to a briefing.

How It Works

The experiment loop. Every iteration: observe metrics, hypothesize (human ideas first), edit train.py, commit to a git branch, train, measure (agent can't see how), keep or revert, log, check convergence.

Hypothesis tracking. Every idea flows through hypotheses.yaml with a novelty guard that blocks duplicates. Detail files record architecture, hyperparameters, expected outcome, actual result, and lineage. Nothing is forgotten between sessions.

Anti-cheating stack. Six structural layers, not prompt-based rules. The agent cannot see evaluate.py, cannot discover scoring formulas, cannot reverse-engineer fixed seeds. It knows the metric name, the direction, and the result. That's it. Research on autonomous ML agents shows that every prompt-based rule got worked around; every code-based rule held.

Two agents, strict boundary. @ml-researcher (Read/Write/Edit/Bash) modifies code and runs experiments. @ml-evaluator (Read/Bash only) analyzes results. An analyst who cannot act on their observations makes more trustworthy observations.

Convergence detection. After N consecutive non-improvements (default 3, configurable), the agent stops. For noisy metrics, /turing:validate auto-configures multi-run evaluation so the agent can't be rewarded for lucky single runs.

Command Reference

Core Loop

Command	What it does
/turing:init [--plan]	Scaffold a new ML project. --plan for literature-grounded research plan.
/turing:train [path] [N]	Run the experiment loop. Auto-detects project from cwd.
/turing:status	Quick status: best model, convergence state
/turing:compare <a> <b>	Side-by-side experiment comparison
/turing:sweep	Systematic hyperparameter sweep

Taste-Leverage Interface