🩻 ai-test-failure-analyzer

Root cause in seconds. Evidence, not intuition.

Feed it a real test result file — Playwright, Jest, Cypress, Newman, k6, or JUnit — and it traces back through your real git history, application logs, and config to surface the actual root cause, with a cited evidence chain and file:line precision. No guesses. No fixture noise. No repeating the obvious.

ai-analyze running 8-phase analysis

🩻 A real analysis — evidence from git, app.log, and .env — no guesses, no fixture noise.

Why ai-test-failure-analyzer

Manual test failure investigation takes 30–60 minutes: open the test output, grep through logs, dig through git history, check recent deploys, ask Slack. And you can still point at the wrong thing — especially when the test file itself has an "intentional failure" comment or a fixture designed to trigger the analyzer.

This tool does it automatically in seconds:

Parses the test result file to extract failing tests with HTTP details
Scans git history for high-risk commits (endpoint renames, migrations, auth changes)
Scans application logs for ERROR/FATAL lines
Reads config files (.env, docker-compose)
Cross-correlates all evidence into clusters
Forms ranked, evidence-cited hypotheses with file:line precision
Never points to test fixtures or "intentional failure" comments as root causes

How it's different

	ai-test-failure-analyzer	Manual triage	Generic LLM
Evidence source	Real git/logs/config	Human memory	Training data
Fixture noise	Blocked by Tier-1 gate	No protection	No protection
`file:line` precision	✅	Sometimes	No
Works without source code	✅ API-only mode	✅	✅
Repeatable	✅	❌	❌
CI-integrated	✅	❌	❌

Supported frameworks

Framework	Format	Command
Playwright	JSON reporter	`playwright test --reporter=json`
Jest / Vitest	JSON	`jest --json --outputFile=results.json`
Cypress	Mochawesome JSON	`cypress run --reporter mochawesome`
pytest	JUnit XML	`pytest --junit-xml=results.xml`
Newman (Postman)	JSON	`newman run col.json --reporters json --reporter-json-export results.json`
k6	Summary JSON	`k6 run --summary-export=results.json script.js`
REST Assured	JUnit XML	standard Maven Surefire output
Any JUnit-compatible	XML	TestNG, Karate, Insomnia CLI

Install

npm (global — JS/CI devs):

npm install -g ai-test-failure-analyzer
ai-analyze analyze playwright-report.json

npx (zero install):

npx ai-test-failure-analyzer analyze playwright-report.json

pipx (Python devs):

pipx install ai-test-failure-analyzer
analyzer analyze playwright-report.json

Claude Code skill:

/plugin install ai-test-failure-analyzer

Install skill to all agents (Claude, Cursor, Codex, Gemini, Windsurf):

ai-analyze install

Usage

CLI

ai-analyze analyze results.json
ai-analyze analyze results.json --mode api-only    # force API-only (no source scan)
ai-analyze analyze results.json --out report.md    # write report to file
ai-analyze analyze results.json --create-issue     # file GitHub issue for top hypothesis

MCP server (Claude Code / Cursor)

Add to your MCP config:

{
  "mcpServers": {
    "ai-test-failure-analyzer": {
      "command": "ai-analyze",
      "args": ["serve-stdio"]
    }
  }
}

Then ask Claude: "Analyze the failures in playwright-report.json"

MCP HTTP (OpenAI / Gemini)

ai-analyze serve-http --port 8765

API-only mode

No source code? No problem. When your workspace has no src/, app/, lib/, or api/ directory — or when you pass --mode api-only — the tool switches to API-only mode.

It analyzes HTTP contract evidence directly from the test results:

ai-test-failure-analyzer

Popularity

What's Inside

README