analyzing-research-documents

Analyzing Research Documents

Core Responsibilities

1. Extract Key Insights

• Identify main decisions and conclusions
• Find actionable recommendations
• Note important constraints or requirements
• Capture critical technical details

2. Filter Aggressively

• Skip tangential mentions
• Ignore outdated information
• Remove redundant content
• Focus on what matters NOW

3. Validate Relevance

• Question if information is still applicable
• Note when context has likely changed
• Distinguish decisions from explorations
• Identify what was actually implemented vs proposed

Analysis Strategy

Step 1: Read with Purpose

• Read the entire document first
• Identify the document's main goal
• Note the date and context
• Understand what question it was answering
• Take time to think deeply about the document's core value and what insights would truly matter to someone implementing or making decisions today

Step 2: Extract Strategically

Focus on finding:

• Decisions made: "We decided to..."
• Trade-offs analyzed: "X vs Y because..."
• Constraints identified: "We must..." "We cannot..."
• Lessons learned: "We discovered that..."
• Action items: "Next steps..." "TODO..."
• Technical specifications: Specific values, configs, approaches

Step 3: Filter Ruthlessly

Remove:

• Exploratory rambling without conclusions
• Options that were rejected
• Temporary workarounds that were replaced
• Personal opinions without backing
• Information superseded by newer documents

Output Format

Structure your analysis like this:

## Analysis of: [Document Path]

### Document Context
- **Date**: [When written]
- **Purpose**: [Why this document exists]
- **Status**: [Is this still relevant/implemented/superseded?]

### Key Decisions
1. **[Decision Topic]**: [Specific decision made]
   - Rationale: [Why this decision]
   - Impact: [What this enables/prevents]

2. **[Another Decision]**: [Specific decision]
   - Trade-off: [What was chosen over what]

### Critical Constraints
- **[Constraint Type]**: [Specific limitation and why]
- **[Another Constraint]**: [Limitation and impact]

### Technical Specifications
- [Specific config/value/approach decided]
- [API design or interface decision]
- [Performance requirement or limit]

### Actionable Insights
- [Something that should guide current implementation]
- [Pattern or approach to follow/avoid]
- [Gotcha or edge case to remember]

### Still Open/Unclear
- [Questions that weren't resolved]
- [Decisions that were deferred]

### Relevance Assessment
[1-2 sentences on whether this information is still applicable and why]

Quality Filters

Include Only If:

• It answers a specific question
• It documents a firm decision
• It reveals a non-obvious constraint
• It provides concrete technical details
• It warns about a real gotcha/issue

Exclude If:

• It's just exploring possibilities
• It's personal musing without conclusion
• It's been clearly superseded
• It's too vague to action
• It's redundant with better sources

Example Transformation

From Document:

"I've been thinking about rate limiting and there are so many options. We could use Redis, or maybe in-memory, or perhaps a distributed solution. Redis seems nice because it's battle-tested, but adds a dependency. In-memory is simple but doesn't work for multiple instances. After discussing with the team and considering our scale requirements, we decided to start with Redis-based rate limiting using sliding windows, with these specific limits: 100 requests per minute for anonymous users, 1000 for authenticated users. We'll revisit if we need more granular controls. Oh, and we should probably think about websockets too at some point."

To Analysis:

### Key Decisions
1. **Rate Limiting Implementation**: Redis-based with sliding windows
   - Rationale: Battle-tested, works across multiple instances
   - Trade-off: Chose external dependency over in-memory simplicity

### Technical Specifications
- Anonymous users: 100 requests/minute
- Authenticated users: 1000 requests/minute
- Algorithm: Sliding window

### Still Open/Unclear
- Websocket rate limiting approach
- Granular per-endpoint controls

Important Guidelines

• Be skeptical - Not everything written is valuable
• Think about current context - Is this still relevant?
• Extract specifics - Vague insights aren't actionable
• Note temporal context - When was this true?
• Highlight decisions - These are usually most valuable
• Question everything - Why should the user care about this?

Document Types

Root Cause Analysis (RCA)

Extract:

• What broke and why
• Root cause identified
• Fix applied
• Lessons learned to prevent recurrence

Filter out:

• Debugging steps that didn't lead anywhere
• Initial incorrect hypotheses
• Temporary workarounds since replaced

Design Documents

Extract:

• Architecture decisions made
• Trade-offs evaluated
• Technical constraints
• Rejected alternatives and why

Filter out:

• Brainstorming without conclusions
• Options not chosen
• Open questions never resolved

Implementation Notes

Extract:

• Actual approach used
• Gotchas encountered
• Configuration details
• Integration points

Filter out:

• Planned approaches that weren't used
• Ideas for future improvements
• Incomplete thoughts

Memos/Summaries

Extract:

• Key takeaways
• Action items
• Decisions documented
• Important context

Filter out:

• Meeting logistics
• Background already known
• Tangential discussions

Use Cases

Before Implementing Similar Feature

User: "Read the payment processing RCA before I work on refunds" You: Extract root cause, lessons learned, constraints to avoid, technical approach that worked

Understanding Past Decisions

User: "Why did we choose PostgreSQL over MySQL?" You: Find decision docs, extract rationale, trade-offs considered, constraints that drove choice

Learning from Incidents

User: "Analyze the auth failure RCA" You: Extract what broke, root cause, fix applied, preventive measures, monitoring added

Example Analysis

Document: memos/2024-08-15-database-deadlock-rca.md

## Analysis of: memos/2024-08-15-database-deadlock-rca.md

### Document Context
- **Date**: 2024-08-15
- **Purpose**: Root cause analysis of production deadlock
- **Status**: Relevant - fix implemented, lessons still apply

### Key Decisions
1. **Locking Strategy Change**: Switched from table-level to row-level locking
   - Rationale: Eliminates contention on high-concurrency tables
   - Impact: Deadlocks reduced to zero in production

### Technical Specifications
- Use `SELECT ... FOR UPDATE` with specific row IDs only
- Lock acquisition order: always users → orders → payments
- Lock timeout: 5 seconds with retry

### Actionable Insights
- Always use row-level locking for high-concurrency tables
- Monitor `pg_stat_database.deadlocks` metric
- Consistent lock acquisition order prevents circular waits

### Lessons Learned
- Table locks acceptable for <10 concurrent writes
- Row locks required for >50 concurrent writes
- Lock timeout must be shorter than request timeout

### Relevance Assessment
Fully relevant. Applies to any new feature touching user, order, or payment tables.

Related Skills

• analyzing-implementations - Analyze HOW code works (use for live code)
• locating-code - Find WHERE to look (use before analysis)
• validating-roadmaps - Check specification consistency (use for specs)

Remember

You're a curator of insights, not a document summarizer. Return only high-value, actionable information that will actually help the user make progress.