From carl-tools
Scan Claude/Codex session logs to find agent behavior patterns, Toolsmith adoption gaps, repeated frustrations, and candidates for new skills/tools.
How this skill is triggered — by the user, by Claude, or both
Slash command
/carl-tools:agent-log-forensicsThe summary Claude sees in its skill listing — used to decide when to auto-load this skill
Use this when the goal is not one bug fix, but making the human+agent system smarter after observing real sessions.
Use this when the goal is not one bug fix, but making the human+agent system smarter after observing real sessions.
Third-order stance: skip the obvious "agent forgot tool" answer. Ask what hidden loop, missing affordance, brittle instruction, or human workaround made the tool forgettable in the first place. Prefer new mental paths over more reminders.
Default to the last 7 days unless the user names a different range. Include remote hosts explicitly named by the user, commonly vesta.
Prefer Toolsmith's built-in scanner when available:
toolsmith scan-agent-logs --days 7 --max-examples 12
toolsmith opportunities --days 7 --max-examples 8
toolsmith scan-agent-logs --days 7 --remote vesta --max-examples 12
If the scanner is not installed yet, say that and fall back to a lightweight count of recent ~/.claude/projects/**/*.jsonl and ~/.codex/sessions/**/*.jsonl files rather than dumping raw prompts.
Create five sections:
For Toolsmith, classify:
Read/Edit/Write, cat, nl, or broad sed -n on >200-line files.apply_patch on a large file after a search/read could have used anchors, but may still be acceptable.file_skeleton, find_and_anchor, get_function, anchored_read, anchored_edit, symbol_replace.For each repeated pattern, propose one of:
Use this table:
| Signal | Better artifact |
|---|---|
| Same review checklist repeated | Skill |
| Same shell command chain repeated | Script |
| Same repo-specific gotcha repeated | CLAUDE.md / AGENTS.md update |
| Agents know the right thing but forget | Prompt snippet / guardrail |
| Agents cannot inspect evidence cheaply | Toolsmith/MCP feature |
| User manually intervenes after agent loops | Skill or deterministic helper that removes the human bridge |
| Expected telemetry is silent | Logging/instrumentation fix before drawing conclusions |
| Same failure appears across projects | User-level skill plus project opt-out, not per-repo copy-paste |
After listing findings, do one further pass:
Name the proposed path in memorable language, for example "shadow workflow collector", "silence detector", "release exit-door drill", or "jank weather report".
Store reports in:
~/dev/agent-notes/<project>/Include:
Use scripts/collect_toolsmith_scan.py to run local and remote Toolsmith scans and write a combined Markdown report.
remote-host-verifier when comparing a command across local and remote hosts.skill-creator when turning findings into new skills.status-copy-trust-audit when confusing CLI output appears repeatedly in logs.beads-knowledge or the project tracker to preserve hard-won patterns that should not be rediscovered.quick-check --cheap for broad synthesis and a different model's view before writing new skills.Guides creation, editing, and verification of skills for AI coding agents using test-driven development with subagent scenarios. Use when authoring or debugging skills.
npx claudepluginhub carlkibler/agent-skills --plugin ralph-loop