qa-auditor

QA Auditor

An intelligent, LLM-driven QA audit plugin for Claude Code that evaluates web applications like a senior QA engineer.

Unlike traditional E2E tests that check pass/fail, QA Auditor evaluates quality across 6 dimensions, finds edge cases, and produces a professional HTML report with star ratings, severity-graded issues, and embedded screenshots.

Installation

claude plugin install github:andreherful/qa-auditor

Requirement: The playwright plugin must be installed:

claude plugin install playwright

Quick Start

1. Create a config file

Create qa-audit.yaml in your project root:

project:
  name: "My App"
  version: "1.0.0"

environment:
  base_url: "http://localhost:3000"

auth:
  strategy: form
  login_url: "/login"
  credentials:
    username_field: "email"
    password_field: "password"
    username: "[email protected]"
    [REDACTED:Password]

features:
  - id: homepage
    name: "Homepage"
    priority: high
    pages:
      - url: "/"
        name: "Landing Page"

2. Run the audit

# Full audit
/qa-audit

# Audit a specific feature
/qa-audit --feature auth

# Custom config path
/qa-audit --config path/to/qa-audit.yaml

# Skip cross-cutting checks
/qa-audit --skip-cross-cutting

3. View the report

The audit generates a self-contained HTML report at:

qa-audit-results/qa-audit-YYYY-MM-DD-HHmmss.html

Open it in any browser — all screenshots are embedded, no external dependencies.

What It Tests

6 Quality Dimensions (scored 1-5)

Dimension	What It Evaluates
Functionality	Do features work correctly? Are workflows complete?
Quality	AI output quality, data accuracy, content relevance
Edge Cases	Empty states, long inputs, special characters, concurrent actions
Integration	API reliability, data flow between components, real-time updates
UX/UI	Layout, responsiveness, loading states, error messages, accessibility
Performance	Page load times, interaction responsiveness, resource usage

Issue Severity Levels

• CRITICAL — Feature broken, data loss, security vulnerability
• HIGH — Major functionality impaired, poor user experience
• MEDIUM — Minor issues, cosmetic problems, inconsistencies
• LOW — Suggestions, minor polish items
• INFO — Observations, notes for consideration

Report Features

• Executive summary with overall score and star rating
• Radar chart showing scores across all 6 dimensions
• Per-feature score cards with detailed scenario results
• Filterable issues table by severity
• Embedded screenshots with lightbox viewer
• Cross-cutting audit results (console errors, responsive, performance)
• Print-ready layout with proper page breaks

Configuration Reference

See examples/qa-audit.yaml for a complete example with all options documented.

Config Structure

project:          # Project metadata
environment:      # URLs and environment settings
auth:             # Authentication strategy
features:         # List of features to test
cross_cutting:    # Cross-cutting audit settings

Auth Strategies

• form — Fill login form fields and submit
• token — Set auth token in localStorage/cookie
• none — No authentication needed

Feature Priority

• critical — Must-test, blocks release
• high — Important, test thoroughly
• medium — Standard testing
• low — Basic smoke test only

Auto-Discovery Mode

If no qa-audit.yaml exists, the auditor enters auto-discovery mode:

1. Navigates to the base URL
2. Discovers available pages and features
3. Runs a basic audit on discovered content
4. Generates a report with recommendations

Project Structure

qa-auditor/
├── .claude-plugin/plugin.json    # Plugin manifest
├── commands/qa-audit.md          # /qa-audit slash command
├── agents/feature-tester.md      # Feature testing sub-agent
├── skills/qa-auditor/
│   ├── SKILL.md                  # Skill overview
│   ├── references/
│   │   ├── evaluation-rubric.md  # Scoring criteria
│   │   └── report-schema.md     # JSON data schema
│   ├── scripts/
│   │   └── generate_report.py   # HTML report generator
│   └── assets/
│       └── report-template.html # HTML/CSS/JS template
├── examples/qa-audit.yaml       # Example configuration
├── README.md
└── LICENSE

License

MIT