qa-flake-triage

Reads historical CI test results (JUnit XML or vendor JSON) and predicts which currently-green tests are likely to go flaky next, using signals from the 8-pattern catalog (test size correlation, async waits with fixed sleeps, parallel-execution heuristics). Returns a ranked watchlist with rationale per test. Use proactively as a weekly screen across a large suite to focus prevention effort before the test starts failing.

Runs a target end-to-end test N times under varied conditions (worker isolation, test order, viewport, network throttling, parallelism) to identify the axis along which the flake reproduces. Returns a probable root cause classified against the 8 flake patterns plus a numeric reproduction rate per axis. Use when a test has been flagged flaky and the team needs to know which condition triggers the failure.

Generates a periodic (weekly / monthly) test-suite health report from CI history - total runs, suite duration, flakiness rate, top failing tests, time-to-green per PR, week-over-week deltas. Emits a markdown summary suitable for a team Slack channel or wiki page. Use as a scheduled CI job to keep test health visible.

Inspects a test suite that flakes under parallel execution and identifies the specific shared state - DB rows, env vars, files, ports, lockfiles, or global module state - that workers are colliding on. Runs targeted instrumentation around suspect resources, correlates each test's writes with another worker's reads, and reports the colliding resource with file:line evidence. Use after `e2e-flake-bisector` has implicated parallel execution.

Orchestrates `git bisect` against a target test or build script to identify the introducing commit of a regression. Wraps the bad/good marking, the `git bisect run` script, the 125 exit code for unbuildable revisions, and the final culprit report. Use when a test that previously passed has started failing 100% of the time on the trunk.

Builds a persistent flakiness infrastructure dashboard from JUnit XML or JSON CI run history: defines the flake-rate metric (failures per test over a configurable window), authors the data model, generates a Grafana time-series panel JSON or configures a Datadog CI Visibility view, derives the quarantine-candidate query, and wires trend alerts. Use when a team needs a long-lived observability surface for test reliability that outlasts any single weekly report.

Reference catalog of flake patterns - async/timing, test ordering, shared parallel state, resource leaks, network, locator drift, environment variance, randomness - with detection heuristics and remediation per pattern. Use when triaging an unknown flake to identify the category before bisecting.

Provides concrete code-level fixes for each of the eight recurring flake patterns cataloged in flake-pattern-reference - replacing fixed sleeps with framework auto-waits, isolating state in beforeEach fixtures, adopting stable role-based locators, mocking network and clock, seeding RNG, and closing leaked resources. Use when a flake has been classified by pattern and the engineer needs the specific code change to apply.

Builds a quarantine workflow for flaky tests - marks the test with the framework's skip/fixme/retry annotation, records the failure-rate observation and a bisect link in the annotation body, sets an auto-expiry date, and produces a CI report listing every quarantined test that has expired and needs re-evaluation. Use when a flaky test is blocking the trunk and must be removed from the gating path without losing track of it.