Validation Frameworks
Frameworks for validating problems, solutions, and product assumptions before committing to full development.
When to Use This Skill
Auto-loaded by agents:
research-ops - For problem/solution validation and assumption testing
Use when you need:
- Validating product ideas before building
- Testing assumptions and hypotheses
- De-risking product decisions
- Running MVP experiments
- Validating problem-solution fit
- Testing willingness to pay
- Evaluating technical feasibility
Core Principle: Reduce Risk Through Learning
Building the wrong thing is expensive. Validation reduces risk by answering critical questions before major investment:
- Problem validation: Is this a real problem worth solving?
- Solution validation: Does our solution actually solve the problem?
- Market validation: Will people pay for this?
- Usability validation: Can people use it?
- Technical validation: Can we build it?
The Validation Spectrum
Validation isn't binary (validated vs. not validated). It's a spectrum of confidence:
Wild Guess → Hypothesis → Validated Hypothesis → High Confidence → Proven
Early stage: Cheap, fast tests (low confidence gain)
Later stage: More expensive tests (high confidence gain)
Example progression:
- Assumption: "Busy parents struggle to plan healthy meals"
- Interview 5 parents → Some validation (small sample)
- Survey 100 parents → More validation (larger sample)
- Prototype test with 20 parents → Strong validation (behavior observed)
- Launch MVP, track engagement → Very strong validation (real usage)
- Measure retention after 3 months → Proven (sustained behavior)
Problem Validation vs. Solution Validation
Problem Validation
Question: "Is this a problem worth solving?"
Goal: Confirm that:
- The problem exists
- It's painful enough that people want it solved
- It's common enough to matter
- Current solutions are inadequate
Methods:
- Customer interviews (discovery-focused)
- Ethnographic observation
- Surveys about pain points
- Data analysis (support tickets, analytics)
- Jobs-to-be-Done interviews
Red flags that stop here:
- No one cares about this problem
- Existing solutions work fine
- Problem only affects tiny niche
- Pain point is mild annoyance, not real pain
Output: Problem statement + evidence it's worth solving
See assets/problem-validation-canvas.md for a ready-to-use framework.
Solution Validation
Question: "Does our solution solve the problem?"
Goal: Confirm that:
- Our solution addresses the problem
- Users understand it
- Users would use it
- It's better than alternatives
- Value proposition resonates
Methods:
- Prototype testing
- Landing page tests
- Concierge MVP (manual solution)
- Wizard of Oz (fake backend)
- Pre-sales or waitlist signups
Red flags:
- Users don't understand the solution
- They prefer their current workaround
- Would use it but not pay for it
- Solves wrong part of the problem
Output: Validated solution concept + evidence of demand
See assets/solution-validation-checklist.md for validation steps.
The Assumption-Validation Cycle
1. Identify Assumptions
Every product idea rests on assumptions. Make them explicit.
Types of assumptions:
Desirability: Will people want this?
- "Users want to track their habits"
- "They'll pay $10/month for this"
- "They'll invite their friends"
Feasibility: Can we build this?
- "We can process data in under 1 second"
- "We can integrate with their existing tools"
- "Our team can build this in 3 months"
Viability: Should we build this?
- "Customer acquisition cost will be under $50"
- "Retention will be above 40% after 30 days"
- "We can reach 10k users in 12 months"
Best practice: Write assumptions as testable hypotheses
- Not: "Users need this feature"
- Yes: "At least 60% of users will use this feature weekly"
2. Prioritize Assumptions to Test
Not all assumptions are equally important. Prioritize by:
Risk: How uncertain are we? (Higher risk = higher priority)
Impact: What happens if we're wrong? (Higher impact = higher priority)
Prioritization matrix:
| Risk/Impact | High Impact | Low Impact |
|---|
| High Risk | Test first | Test second |
| Low Risk | Test second | Maybe skip |
Riskiest assumptions (test these first):
- Leap-of-faith assumptions the entire business depends on
- Things you've never done before
- Assumptions about user behavior with no data
- Technical feasibility of core functionality
Lower-risk assumptions (test later or assume):
- Based on prior experience
- Easy to change if wrong
- Industry best practices
- Minor features
3. Design Validation Experiments
For each assumption, design the cheapest test that could prove it wrong.
Experiment design principles:
1. Falsifiable: Can produce evidence that assumption is wrong
2. Specific: Clear success/failure criteria defined upfront
3. Minimal: Smallest effort needed to learn
4. Fast: Results in days/weeks, not months
5. Ethical: Don't mislead or harm users
The Experiment Canvas: See assets/validation-experiment-template.md
4. Run the Experiment
Before starting:
During:
- Track metrics rigorously
- Document unexpected learnings
- Don't change experiment mid-flight
After:
- Analyze results honestly (avoid confirmation bias)
- Document what you learned
- Decide: Pivot, persevere, or iterate
Validation Methods by Fidelity
Low-Fidelity (Hours to Days)
1. Customer Interviews
- Cost: Very low (just time)
- Validates: Problem existence, pain level, current solutions
- Limitations: What people say ≠ what they do
- Best for: Early problem validation
2. Surveys
- Cost: Low
- Validates: Problem prevalence, feature preferences
- Limitations: Response bias, can't probe deeply
- Best for: Quantifying what you learned qualitatively
3. Landing Page Test
- Cost: Low (1-2 days to build)
- Validates: Interest in solution, value proposition clarity
- Measure: Email signups, clicks to "Get Started"
- Best for: Demand validation before building
4. Paper Prototypes
- Cost: Very low (sketch on paper/whiteboard)
- Validates: Concept understanding, workflow feasibility
- Limitations: Low realism
- Best for: Very early solution concepts
Medium-Fidelity (Days to Weeks)
5. Clickable Prototypes
- Cost: Medium (design tool, 2-5 days)
- Validates: User flow, interaction patterns, comprehension
- Tools: Figma, Sketch, Adobe XD
- Best for: Usability validation pre-development
6. Concierge MVP
- Cost: Medium (your time delivering manually)
- Validates: Value of outcome, willingness to use/pay
- Example: Manually curate recommendations before building algorithm
- Best for: Validating value before automation
7. Wizard of Oz MVP
- Cost: Medium (build facade, manual backend)
- Validates: User behavior, feature usage, workflows
- Example: "AI" feature that's actually humans behind the scenes
- Best for: Testing expensive-to-build features
High-Fidelity (Weeks to Months)
8. Functional Prototype
- Cost: High (weeks of development)
- Validates: Technical feasibility, actual usage patterns
- Limitations: Expensive if you're wrong
- Best for: After other validation, final pre-launch check
9. Private Beta
- Cost: High (full build + support)
- Validates: Real-world usage, retention, bugs
- Best for: Pre-launch final validation with early adopters
10. Public MVP
- Cost: Very high (full product)
- Validates: Product-market fit, business model viability
- Best for: After all other validation passed
Setting Success Criteria
Before running experiment, define what success looks like.
Framework: Set three thresholds
- Strong success: Clear green light, proceed confidently
- Moderate success: Promising but needs iteration
- Failure: Pivot or abandon
Example: Landing page test
- Strong success: > 30% email signup rate
- Moderate success: 15-30% signup rate
- Failure: < 15% signup rate
Example: Prototype test
- Strong success: 8/10 users complete task, would use weekly
- Moderate success: 5-7/10 complete, would use monthly
- Failure: < 5/10 complete or no usage intent
Important: Decide criteria before seeing results to avoid bias.
Teresa Torres Continuous Discovery Validation
Opportunity Solution Trees
Map opportunities (user needs) to solutions to validate:
Outcome
└── Opportunity 1
├── Solution A
├── Solution B
└── Solution C
└── Opportunity 2
└── Solution D
Validate each level:
- Outcome: Is this the right goal?
- Opportunities: Are these real user needs?
- Solutions: Will this solution address the opportunity?
Assumption Testing
For each solution, map assumptions:
Desirability assumptions: Will users want this?
Usability assumptions: Can users use this?
Feasibility assumptions: Can we build this?
Viability assumptions: Should we build this?
Then test riskiest assumptions first with smallest possible experiments.
Weekly Touchpoints
Continuous discovery = continuous validation:
- Weekly customer interviews (problem + solution validation)
- Weekly prototype tests with 2-3 users
- Weekly assumption tests (small experiments)
Goal: Continuous evidence flow, not one-time validation.
See references/lean-startup-validation.md and references/assumption-testing-methods.md for detailed methodologies.
Common Validation Anti-Patterns
1. Fake Validation
What it looks like:
- Asking friends and family (they'll lie to be nice)
- Leading questions ("Wouldn't you love...?")
- Testing with employees
- Cherry-picking positive feedback
Fix: Talk to real users, ask open-ended questions, seek disconfirming evidence.
2. Analysis Paralysis
What it looks like:
- Endless research without decisions
- Testing everything before building anything
- Demanding statistical significance with 3 data points
Fix: Accept uncertainty, set decision deadlines, bias toward action.
3. Confirmation Bias
What it looks like:
- Only hearing what confirms existing beliefs
- Dismissing negative feedback as "they don't get it"
- Stopping research when you hear what you wanted
Fix: Actively seek disconfirming evidence, set falsifiable criteria upfront.
4. Vanity Validation
What it looks like:
- "I got 500 email signups!" (but 0 conversions)
- "People loved the demo!" (but won't use it)
- "We got great feedback!" (but all feature requests, no usage)
Fix: Focus on behavior over opinions, retention over acquisition.
5. Building Instead of Validating
What it looks like:
- "Let's build it and see if anyone uses it"
- "It'll only take 2 weeks" (takes 2 months)
- Full build before any user contact
Fix: Always do cheapest possible test first, build only after validation.
Validation Checklist by Stage
Idea Stage
Concept Stage
Pre-Build Stage
MVP Stage
Ready-to-Use Resources
In assets/:
- problem-validation-canvas.md: Framework for validating problems before solutions
- solution-validation-checklist.md: Step-by-step checklist for solution validation
- validation-experiment-template.md: Design experiments to test assumptions
In references/:
- lean-startup-validation.md: Build-Measure-Learn cycle, MVP types, pivot decisions
- assumption-testing-methods.md: Comprehensive assumption testing techniques