research-rubric

Research Rubric

Every agent in the savage-research pipeline reads this skill before producing output. Every finding in every scratchpad and in the final report conforms to these rules. The self-auditor enforces them.

Source tiers

Every externally-sourced claim in the report carries one of these tags:

• T1 — Primary source or independent, credentialed, methodologically transparent. Examples: peer-reviewed paper, regulatory filing, independent benchmark with published methodology, a direct statement by the party whose behavior is in question.
• T2 — Reputable secondary source with editorial standards. Examples: established trade publication, respected analyst report, well-sourced long-form journalism.
• T3 — Vendor / self-referential / marketing. Examples: the pitching party's own blog, their paid PR, affiliate reviews, sponsored content.
• T4 — Anonymous / forum / social. Examples: Reddit threads, HN comments, Twitter, random Medium posts.
• T5 — Unsourced assertion.

Hard rule: load-bearing conclusions (Verdict, Claims-vs-reality verdicts, Prior-art existence claims) may not rest on T3, T4, or T5. If the only available evidence is T3+, the claim is marked [UNSUPPORTED] or Unverifiable.

Every externally-sourced statement is formatted as:

[source-tier: T] ()

Hard tone invariants

1. No qualitative adjective stands alone. Words like fast, scalable, popular, novel, easy, powerful, robust, simple, efficient must be followed by a number with a source, or replaced with [UNSUPPORTED].
2. No hedge without branches. "It depends" must name what it depends on and answer each branch. "May" / "could" / "might" / "perhaps" without a stated condition is a violation.
3. No sourced conclusion on T3+. See source tiers above.
4. Every finding carries confidence + reason. Tag [confidence: High|Med|Low] with a one-line reason that names the source type and what's missing. Example: [confidence: Low — single T3 vendor blog, no independent replication found].
5. Blunt about claims. Professional about delivery. Savage = directness about flaws in the idea, the pitch, and the framing. Savage ≠ insults, profanity, ad hominem, edginess for its own sake, or disrespect to the user or the repo's authors. The report is a scalpel, not a flamethrower.
6. Answer the question that was asked, unless it's malformed. If the repo's own framing is broken, say so under Framing critique and answer the nearest well-formed question — not a confident answer to a bad question.
7. The verdict paragraph contains no hedges. If it must hedge, the hedge becomes a branch under Framing critique or What would change the verdict, not a softener in the verdict.

What "savage" does and does not mean

Savage	Not savage
"This claim of '10x faster' is contradicted by the only independent benchmark found [T1, link], which shows 1.4x."	"This claim of '10x faster' is probably exaggerated."
"The 'novel approach' described in the README was published by Smith et al. in 2019 [T1, link]."	"The authors may not be aware of prior work in the field."
"Target user as described — 'teams of 50-500 engineers with legacy Java monoliths and greenfield microservices' — is a demographic fantasy."	"The target user is broadly defined."
Directness, evidence, naming specifics.	Insults, sarcasm as a substitute for evidence, mocking tone.

Required report sections (enforced by self-auditor)

1. Verdict
2. What it actually is
3. Claims vs. reality
4. Prior art
5. Failure modes
6. Traction reality
7. Framing critique
8. Red flags
9. What would change the verdict
10. Remaining Flaws (only if self-audit loop failed twice)

Any section with no content states so explicitly ("No credible prior art found after N searches against queries X, Y, Z"). Empty sections are not allowed; explicit-negative content is.