From braintrust-skills
Use when analyzing and reducing Braintrust usage costs with the bt CLI, including logs/GB ingest, scorers, Topics, Gateway LLM spend, datasets, experiments, attachments, or large trace payloads.
How this skill is triggered — by the user, by Claude, or both
Slash command
/braintrust-skills:bt-cost-optimizerThe summary Claude sees in its skill listing — used to decide when to auto-load this skill
Find what is driving Braintrust usage cost, separate measured evidence from heuristics, and recommend safe changes to logging, scoring, Topics, Gateway, datasets, and experiments.
Find what is driving Braintrust usage cost, separate measured evidence from heuristics, and recommend safe changes to logging, scoring, Topics, Gateway, datasets, and experiments.
JSONAttachment, scorer filtering, Topics, Gateway caching, provider routing, retention, or code changes would reduce costLoad these as needed:
references/braintrust-cost-model.md - pricing/resource accounting facts and evidence limitsreferences/bt-query-patterns.md - bt commands and SQL patterns for collecting samplesreferences/optimization-patterns.md - concrete log, scorer, Topics, Gateway, and experiment recommendationsreferences/report-template.md - expected report structurescripts/analyze-cost-drivers.py - local analyzer for exported Braintrust rowsscripts/fetch-project-config.py - fetches scorer definitions, automation rules (sampling rates, filters, scope), experiments, datasets, and project settings via the REST API and bt CLIbt evidence. Run bounded queries and inspect a row sample before recommending changes.https://www.braintrust.dev/docs/guides/logs/score) or in data returned by bt commands. If you cannot verify it, say so explicitly rather than naming it.JSONAttachment for large JSON that must remain available for debugging but does not need search, filtering, scorer logic, or dashboard grouping.JSONAttachment is a guaranteed billable-byte reduction. Use it to reduce indexed trace bodies and improve span-size/UI behavior; pair it with actual byte reduction when the goal is cost.Resolve the active Braintrust context before querying.
bt status --json
cat .bt/config.json 2>/dev/null
bt projects list --json
Use .bt/config.json first when present. If no project is configured, ask the user for the project name, then resolve the project ID with bt projects list --json.
Capture:
Load references/bt-query-patterns.md. Pull bounded evidence before making recommendations.
Minimum evidence set:
bt sql --json "SELECT * FROM project_logs('<PROJECT_ID>', shape => 'spans') WHERE created >= NOW() - INTERVAL 7 day LIMIT 1000" > /tmp/bt-cost-project-spans.json
bt sql --json "SELECT COUNT(*) AS spans, COUNT(DISTINCT root_span_id) AS traces FROM project_logs('<PROJECT_ID>', shape => 'spans') WHERE created >= NOW() - INTERVAL 7 day" > /tmp/bt-cost-volume.json
bt scorers list --json > /tmp/bt-cost-scorers.json
bt topics status --json > /tmp/bt-cost-topics-status.json
bt topics config --json > /tmp/bt-cost-topics-config.json
Also run fetch-project-config.py to pull scorer definitions cross-joined with automation rules, plus experiment/dataset counts and project settings:
python3 skills/bt-cost-optimizer/scripts/fetch-project-config.py \
--project-id <PROJECT_ID> \
--output /tmp/bt-config-report.md \
--json-output /tmp/bt-config-summary.json
This requires BRAINTRUST_API_KEY for automation rules, experiments, datasets, and project settings. If the key is not set, scorer definitions are still fetched via bt scorers list. When automation rules are unavailable, note that sampling rates and filters must be reviewed in the Braintrust UI (Project → Logs → Score) before making any sampling recommendations.
If the project is very high volume, reduce the sample limit and use targeted queries for root spans, scorer spans, LLM spans, or known trace IDs.
When experiments or datasets are suspected, sample those too with experiment('<EXPERIMENT_ID>') or dataset('<DATASET_ID>') if the IDs are known.
Run the analyzer on one or more exported files.
python3 skills/bt-cost-optimizer/scripts/analyze-cost-drivers.py \
/tmp/bt-cost-project-spans.json \
--sample-days 7 \
--output bt-cost-optimization-report.md \
--json-output bt-cost-optimization-summary.json
If no sample window is known, omit --sample-days; the report will still rank fields, rows, traces, scorer spans, and token totals but will not extrapolate monthly usage.
Find where cost drivers are produced. Prefer targeted code search over broad assumptions.
rg -n "init_logger|initLogger|logger\\.log|span\\.log|start_span|startSpan|braintrust|JSONAttachment|Attachment\\(" .
rg -n "messages|transcript|documents|chunks|retrieved|context|prompt|completion|tool_calls|response|request|metadata|embedding" .
rg -n "sampling_rate|apply_to_root_span|apply_to_span_names|btql_filter|LLM-as-a-Judge|autoevals|scorer" .
rg -n "x-bt-use-cache|x-bt-cache-ttl|x-bt-endpoint-name|x-bt-compress-audio|BraintrustGateway|braintrust_proxy|base_url" .
Look for:
btql_filter, with high sampling_rate, or with scorer logging enabled when scorer traces are not neededLoad references/optimization-patterns.md and write bt-cost-optimization-report.md. Every recommendation must include:
bt sql, bt view, bt topics, bt scorers, code search, billing/UI config, or assumptionUse these decision rules:
JSONAttachment or an external reference and keep summary metadata inline.Do not edit customer instrumentation by default. The default output is an evidence-backed report plus optional patch recommendations.
Only make code changes when the user asks. When changing code:
Write:
bt-cost-optimization-report.md - human-readable findings and recommendationsbt-cost-optimization-summary.json - machine-readable summary when using the analyzer or automationIf evidence is insufficient, say exactly what is missing and provide the next bt command or config location to collect it.
Creates, edits, and optimizes skills for Claude Code, including drafting, evaluating with test prompts, iterating on performance, and improving skill descriptions for better triggering accuracy.
npx claudepluginhub dpguthrie/braintrust-skills --plugin bt-flywheel