spec-quality

Specification Quality

Criteria and tools for evaluating specification quality.

Applicability Rubric

Condition	Pass	Fail
Spec review requested	Need to assess spec quality	No review needed
Pre-implementation check	Spec about to be implemented	Already implemented
Quality gate	Spec must meet minimum bar before proceeding	Quality already verified
Improvement cycle	Iterating on spec after feedback	No prior version to compare against

Apply when: Any condition passes

Core Principles

Specification Rubric

Rate each item Y (yes) or N (no).

Intent Layer

#	Criterion	Required	Y/N
1	Purpose stated in one sentence?	✓
2	Target users identified?	✓
3	Impacts (behavior changes) identified?
4	Success criteria measurable or verifiable?

Design Layer

#	Criterion	Required	Y/N
5	Each documented feature has defined behavior?	✓
6	Error scenarios cover all documented features?	✓
7	Interaction points (internal and external) have explicit contracts?
8	Implementer can build without clarifying questions?	✓

Consistency Layer

#	Criterion	Required	Y/N
9	Key terms defined and used consistently throughout?
10	Recurring situations have named patterns?
11	Two implementers would produce compatible results?

Evidence Criteria for Y/N scoring:

Score based on the spec's declared scope — what the spec claims to cover. Do not score N because the spec omits something outside its declared scope.

#	Score Y when	Score N when
1	Spec contains an explicit sentence stating what problem the system solves	No purpose statement, or purpose restates the feature name ("A login system for logging in")
2	Spec names specific user roles or personas	"Users" mentioned generically without identifying who or what they accomplish
3	Spec lists observable behavior changes or measurable outcomes	No impacts listed, or only vague goals ("improve performance")
4	Success criteria use numbers, conditions, or verifiable statements	Unmeasurable criteria ("fast and reliable", "good UX")
5	Every feature the spec lists has at least one defined behavior (action → result)	Features listed as bullet points without behavior definitions
6	Each documented feature has at least one error/failure scenario defined	Missing error handling for documented features, or only generic "handle errors appropriately"
7	Module interaction points described in the spec have explicit input/output contracts	Modules interact without defined interfaces
8	An implementer could build the documented scope without asking clarifying questions	Ambiguous terms, undefined states, or missing decisions within documented scope
9	Key domain terms are defined; each concept has exactly one name throughout	Same concept called different names in different sections
10	Similar situations within the spec reference shared patterns or conventions	Ad-hoc solutions for similar problems in different sections
11	Two implementers would produce compatible results for all documented behaviors	Interpretation would vary for documented scope

Scoring rules:

• Any Required N → must address before implementation
• Non-required N → report as upgrade path only if user targets a higher level

Quality Level Classification

Level	Criteria	Suitable For
Minimal	All Required items Y (#1,2,5,6,8)	Small team, rapid iteration
Usable	All Required Y + #3,4 Y	Standard development
Complete	All 11 items Y	Multi-team, external consumers
Over-specified	All Y but Balance Check fails	Needs trimming of implementation details

Balance Check (for review verification; see spec-principles Specify vs Leave Open for writing decisions)

Question	Design Decision (specify)	Implementation Detail (open)
User-visible?	Error messages, CLI output	Log format, variable names
Affects module contracts?	Interface signatures	Internal functions
Could cause inconsistent interpretation?	Error handling pattern, Business rules	Algorithm choice, Performance optimization

Test: "If implemented differently, would users notice or would modules conflict?"

Warning signs of over-specification: internal implementation details, algorithm choices, or a named technology—unless the choice is user-visible, required for cross-implementer consistency, or carries a binding constraint (security, compliance, interop). A technology named to satisfy a property (e.g. "use bcrypt" for password hashing) should be rewritten as the property ("adaptive hashing, work factor ≥ N") unless that specific technology is itself mandated. See spec-principles Technology Choices for the discriminator test.

Warning signs of under-specification: vague terms ("appropriate", "reasonable"), undefined behavior for reachable states within documented features

Review Scope Principle

Review evaluates the quality of what the spec declares, not what it could declare.

Finding Type	Definition	Report As
Defect	Something present in the spec but incorrect, ambiguous, or dangerous	Must Fix
Stability/Security Gap	Missing handling that would cause system crash, data loss, or security breach within documented features	Suggested Fix
Enhancement Gap	Additional detail that would improve quality but system works without it	Note (only when user targets higher quality level)

Rules:

• Do NOT suggest adding features or scope the spec doesn't claim to cover
• Do NOT flag non-required rubric items as problems unless the user's target quality level requires them
• Do NOT ask "do you want to add X?" for things outside the spec's declared scope
• Report non-required N items as upgrade path: "To reach [Usable/Complete] level, consider: ..."

Common Problems (diagnosis; see spec-principles Anti-Patterns for prevention)

Problem	Symptom	Severity	Fix
Missing intent	Technical tasks instead of user value	Must Fix (Required #1,2)	Add intent layer
Undefined scenarios (stability)	Documented feature lacks error handling; failure would crash or corrupt	Suggested Fix	Add error scenarios for documented features
Undefined scenarios (detail)	Could add more edge cases but system functions without them	Note	Mention only if user targets higher quality
Over-specification	Implementer constrained unnecessarily	Must Fix	Keep only observable behaviors; if a technology is named, confirm it carries a binding constraint, else replace with the property it must satisfy
Inconsistent patterns	Similar problems solved differently	Note	Extract and reference patterns
Inconsistent terminology	Same concept has multiple names	Note	Define key terms, use consistently
Vague language	Ambiguous interpretation within documented scope	Must Fix	Use specific values or criteria
Hidden assumptions	Works only in specific context	Suggested Fix	Make all assumptions explicit
Explanatory notes	"Note: because..." appears	Must Fix	Rewrite as direct statement
Phase / pending markers	"(Future)", "(v2)", TBD, TODO, 暫定, (待討論) in spec body	Must Fix	Decide now; for undecided items use Handling Uncertainty section, not inline markers
Thinking traces	"原本是 X，調整為 Y"; "Considered A, chose B"; inline rationale ("because we chose..."); revision notes	Must Fix	Rewrite as current state; move history/rationale to commit log or ADR

Completion Rubric

Before Quality Assessment

Criterion	Pass	Fail
Full spec read	Entire specification read and understood	Partial or skimmed reading
Declared scope identified	Know what the spec claims to cover (features, boundaries, non-goals)	Evaluating against assumed scope
Target quality level selected	Chose Minimal/Usable/Complete based on context	No target level determined

During Quality Assessment

Criterion	Pass	Fail
All 11 items evaluated	Every rubric item scored Y or N	Items skipped or unclear
Evidence for each N	Each N has specific evidence from spec	N without justification
Balance Check completed	Over/under-specification assessed	Balance not checked
Common Problems scanned	All 9 problem types checked	Problems check incomplete

After Quality Assessment

Criterion	Pass	Fail
Findings classified by severity	Must Fix, Suggested Fix, and Note separated clearly	Flat unprioritized list
Scope respected	No suggestions to add features or scope beyond what spec declares	Recommending additions outside declared scope
Actionable recommendations	Each finding has a specific fix suggestion	Vague improvement advice
Clear verdict	Quality level determined (Minimal/Usable/Complete/Over-specified)	No clear assessment