Holdfast
Governed evolution for LLM and agentic systems.
The destination is fixed. The route gets better.
Holdfast separates what your downstream systems depend on (frozen) from how you deliver it (evolvable), then uses evidence to improve the route while guaranteeing the destination doesn't change. Built for teams running LLM pipelines, agentic workflows, and prompt-driven systems that need to improve without regressing.
Built for Claude Code & Python pipelines
Claude Code — no Python required
Install the holdfast skill and it works out of the box. No Python library needed.
Use Claude Code the way you already do — code reviews, test generation, refactoring, planning. Holdfast tracks your interactions, detects when quality drifts, and proposes improvements to Claude's own approach.
- Discover — holdfast notices repeatable tasks and offers to track them
- Track — after each tracked interaction, holdfast silently logs evidence (outcome, quality, what worked)
- Interpret — ask Claude what's drifting and it reads the accumulated evidence, runs detection, explains patterns
- Evolve — Claude proposes a targeted change to its own approach, you approve, the approach improves
Tell Claude to start tracking:
"Use holdfast to track my code reviews."
Or add it to your CLAUDE.md:
Use holdfast to track the quality of code reviews in this project.
Claude sets up the contract, starts logging evidence silently, and surfaces insights when patterns emerge.
Python pipelines
The Python library instruments your pipeline and collects evidence. Wire @track into your functions, and holdfast logs every run, validates against invariants, and detects drift — giving Claude (or your own LLM) the data it needs to propose improvements.
from holdfast import Contract, track
contract = Contract.load("holdfast/contracts/my-pipeline/")
@track(contract)
async def classify(item: dict) -> dict:
prompt = contract.get_evolvable("prompt")
return await your_llm_call(prompt, item)
# Every call logs evidence. Pass/fail determined by invariant validation.
# Evolution proposals are backed by this accumulated evidence.
For CI or fully automated pipelines, propose_evolution() drives evolution programmatically without Claude Code.
What it looks like
A typical Claude Code session reviewing evidence across contracts:
You: How's my classifier doing?
Claude: Here's what the holdfast evidence shows:
Classifier — English (847 runs)
Drift alert (triggered): Mean confidence shifted from baseline 0.82 → 0.71
(shift 0.11, threshold 0.10). Confidence is trending lower — the model is
less certain on recent inputs. Could be input distribution shift or the
prompt change from last week.
Pass rate: Baseline (first 50) = 88% → Recent (last 50) = 94%. Accuracy
improved despite lower confidence — the model is better calibrated now.
1 volatile input category: product reviews with mixed sentiment (stdev 0.31,
scores [0.9, 0.4, 0.6]). The rest are stable within ±0.1.
Classifier — Spanish (312 runs)
No alerts. Consistent output across all detection windows.
Summarizer — Short Form (1,203 runs)
Failure rate alert: 23% in last 50 runs (threshold 15%). Failures are all
schema violations — summaries exceeding the 280-char limit. The model is
producing longer outputs since the prompt was last evolved.
Bottom line
The classifier drift is minor — watch it over the next run cycle. The
summarizer needs attention: the char limit failures are climbing.
You: Propose an evolution for the summarizer.
Claude: Based on runs 01180, 01187, 01192, 01195, 01201 — all over the char limit:
Proposed change to evolvable/prompt.md:
+ Your summary MUST be under 280 characters. Count carefully.
+ If the input is complex, prioritize the key finding over completeness.
This preserves the frozen output schema and targets the specific failure
pattern. Want me to apply it?
You: Yes.
Claude: Applied evolution evo-00003 (v2 → v3). Previous version snapshotted —
rollback to v2 anytime.
When to use holdfast
Holdfast fits any repeatable LLM or agent task that needs to improve over time without regressing — whether the output feeds downstream systems or is the end product itself.
Claude Code tasks (no Python needed):
- Code reviews — track whether reviews catch real bugs, flag false positives, miss security issues
- Test generation — track whether tests cover edge cases, catch real bugs, run without modification
- Planning and refactoring — track whether plans lead to successful implementations, whether refactors break things
- Documentation and auditing — track consistency and completeness over time
