pr-review-extract-lessons

Extract PR Lessons

Analyze completed PR review plan artifacts to find recurring mistake patterns, accumulate candidates across PRs, and propose rules for CLAUDE.md when a pattern repeats. This skill implements a feedback loop: review comments from automated tools (Copilot, CodeRabbit) become project-level rules that prevent the same class of mistake from recurring.

Prerequisites

• A completed plan artifact in .pr-review/ (any status — it reads the triage data, not the posting state)

Worktree awareness

This skill often runs inside a git worktree which is ephemeral — it gets deleted after the branch is merged. Lessons and rules must be written to the main repo, not the worktree.

At the start of this skill, detect if you're in a worktree:

MAIN_REPO=$(git rev-parse --path-format=absolute --git-common-dir | sed 's|/\.git$||')
CURRENT_DIR=$(pwd)
if [ "$MAIN_REPO" != "$(git rev-parse --show-toplevel)" ]; then
    echo "WORKTREE detected. Main repo: $MAIN_REPO"
else
    echo "Not a worktree. Main repo: $CURRENT_DIR"
fi

• Read .pr-review/ plan files from the current directory (worktree or main — wherever the plan was written)
• Write CLAUDE.md changes to $MAIN_REPO/CLAUDE.md — never to the worktree copy
• Write candidate file to $MAIN_REPO/.claude/pr-lesson-candidates.md — per-project, in the main repo, never in a worktree

How the 2-PR threshold works

A single PR's review isn't enough signal to create a project rule — it could be a one-off mistake or a reviewer quirk. Instead:

1. First occurrence: a pattern is added to $MAIN_REPO/.claude/pr-lesson-candidates.md as a candidate at 1/2
2. Second occurrence (different PR): the candidate is promoted to 2/2 and proposed for CLAUDE.md
3. User approves: rule is added to CLAUDE.md, candidate is removed
4. User rejects: candidate is removed entirely (don't keep nagging)

This prevents overfitting on a single PR while still capturing genuine recurring issues.

Step 1: Load the plan artifact

Find the most recent plan artifact in .pr-review/ (or use the PR number the user specifies). Extract all threads that were classified as apply_suggestion or fix — these represent actual mistakes that needed correction.

If there are fewer than 2 actionable threads, report "Not enough signal — only N actionable thread(s). Skipping analysis." and stop.

Step 2: Identify mistake categories

Group the actionable threads by root cause. Don't just restate the reviewer's comment — ask why the mistake was made:

• Missing error handling → agent didn't consider failure modes
• Wrong Kotlin idiom → agent unfamiliar with language conventions
• Missing docstrings → agent didn't follow existing patterns in the codebase
• Incorrect API usage → agent didn't read the API docs carefully
• etc.

Only keep categories with ≥2 threads in this PR. A single instance of a mistake isn't a pattern.

Step 3: Check CLAUDE.md

Read the project's CLAUDE.md file. For each surviving pattern, check if a rule already covers it. If so, skip — don't duplicate.

For example, if CLAUDE.md already says "Always add KDoc to public methods" and the pattern is "missing documentation on public APIs", that's already covered.

Step 4: Check and update the candidate file

The candidate file lives at $MAIN_REPO/.claude/pr-lesson-candidates.md. If it doesn't exist yet, create it with this header:

# PR Lesson Candidates

Accumulated patterns from PR reviews. Rules are promoted to project CLAUDE.md
after appearing in 2+ separate PRs. Managed by /pr-review:pr-review-extract-lessons.

For each surviving pattern (not already in CLAUDE.md):

If not yet a candidate

Add a new entry:

## Candidate: <short pattern name>
- Proposed rule: "<the generic, actionable rule>"
- First seen: PR #<N> (<date>) — <count> threads (<brief description>)
- Promotion count: 1/2

If already a candidate at 1/2

Check if it's from a different PR (same PR shouldn't count twice). If different PR, update:

- Seen again: PR #<N> (<date>) — <count> thread(s)
- Promotion count: 2/2 → **ready for promotion**

Step 5: Propose promotions

For each candidate at 2/2:

1. Draft the rule. It must be:

   • Broadly applicable — not tied to a specific function name, variable, or file path
   • Actionable — tells the agent what to do or avoid in concrete terms
   • Self-contained — understandable without reading any specific PR

   Bad (too specific): "Always add a null check in processFrame() after calling getBuffer()" Good (generic): "Always null-check return values from methods that acquire hardware resources — they can fail silently under contention."

   Bad (too vague): "Think carefully about edge cases" Good (actionable): "When adding error handling, check for both synchronous exceptions and callback-delivered errors — Camera2 surfaces failures through both paths."

2. Show the user the proposed rule and which PRs triggered it.

3. Ask for approval. Do not edit CLAUDE.md without explicit confirmation.

4. On approval: append the rule to the "Rules for AI Agents" section of CLAUDE.md in the main repo (use $MAIN_REPO/CLAUDE.md, not the worktree copy). Remove the candidate from $MAIN_REPO/.claude/pr-lesson-candidates.md.

5. On rejection: remove the candidate from $MAIN_REPO/.claude/pr-lesson-candidates.md. Don't re-propose it.

Step 6: Report to user

Present a summary of what happened:

## Lessons from PR #<N>

**Patterns observed:** <count> mistake categories found
**Already covered by CLAUDE.md:** <count> (skipped)
**New candidates added (1/2):** <count>
  - "<pattern name>" — <brief description>
**Promoted to CLAUDE.md (2/2):** <count>
  - "<rule text>"
**No pattern found:** <only if zero patterns survived filtering>

Takeaway: <one sentence summarizing the most significant insight, or "No recurring patterns yet — will check again after the next review.">

Important guidelines

• Never auto-edit CLAUDE.md. Always ask for user approval.
• Require ≥2 threads per pattern per PR. A single comment isn't a pattern.
• Require 2 separate PRs before promotion. This prevents overfitting on one review.
• Reject overly specific rules. If a proposed rule mentions a specific function name, variable name, or file path, it's too narrow. Generalize it.
• Reject overly vague rules. "Be careful" and "think about edge cases" aren't actionable. If you can't write a concrete rule, skip.
• Same-PR duplicates don't count. If a candidate was created from PR #42 and we're analyzing PR #42 again (e.g., re-run), don't increment.
• This skill never modifies source files. It only reads plan artifacts and optionally updates CLAUDE.md and the candidate file.