agent-benchmark

Agent-Benchmark

A Claude Code plugin that extracts theoretical physics benchmark test cases from conversation history.

What it does

/extract-testcase parses Claude Code or Codex JSONL conversation logs, identifies physics problems and research tasks within them, and generates structured YAML test cases. It supports:

• Single question test cases (concept, derivation, literature, simulation, transfer)
• Composite test cases for multi-step research tasks with step dependencies

Install

Step 1: Add the marketplace source

claude plugins marketplace add rainshed/Agent-Benchmark

Step 2: Install the plugin

claude plugins install agent-benchmark@agent-benchmark

Usage

/extract-testcase path/to/conversation.jsonl
/extract-testcase session1.jsonl session2.jsonl

The skill will:

1. Parse the JSONL files (auto-detects Claude Code vs Codex format)
2. Identify physics problems and classify them by type and difficulty
3. Generate YAML test cases using the appropriate template
4. Save results to the examples/ directory

Test case templates

Template	Use case
question	Single physics problem (concept, derivation, simulation, etc.)
composite	Multi-step research task with dependencies between steps

Difficulty levels

• L1 — Textbook knowledge, widely discussed in literature
• L2 — Requires simple reasoning from given materials
• L3 — Complex research task requiring multi-step reasoning

Project structure

Agent-Benchmark/
├── .claude-plugin/
│   └── plugin.json
├── skills/
│   └── extract-testcase/
│       └── SKILL.md
├── templates/           # Reference templates
└── tools/               # Standalone conversation parser

License

MIT