fleet-ci-audit

Fleet CI Audit

Use this as the first step of any fleet cleanup. The output is a precise table of what's broken, what's Dependabot noise, and what's waiting for the billing cap to reset.

The full-fleet audit is what the user means when they say "fix all known issues" or "make the fleet green." Without this skill, the agent checks the latest run per repo, sees a wall of red, and starts debugging code that's already correct. The audit separates the real signal from the noise.

The Four Buckets

Every non-success conclusion in the fleet's CI/lint history falls into one of four categories. The order matters — check them in this order.

1. billing_cap — Spending limit needs to be increased

• Signal: The workflow run annotation says The job was not started because recent account payments have failed or your spending limit needs to be increased.
• Why it happens: GitHub Pro (and other paid plans) have a soft cap on monthly Actions minutes (2,000 for Pro). Once exceeded, new runs don't start — they fail at the queue level, not the code level.
• Fix: User (not agent) goes to GitHub Settings → Billing → increase the spending limit. The agent cannot fix this.
• Verify: Wait 1-3 hours, re-trigger with a no-op touch, check if the run goes green.

# Detect: search the run view output for the spending-limit string
gh run view <id> --repo camster91/<repo> | grep -i "spending limit"

2. dependabot_pr — Auto-PR failure on a non-main branch

• Signal: The run is on a branch like dependabot/npm_and_yarn/development-dependencies-<hash>. Event type is pull_request.
• Why it happens: Dependabot auto-generates PRs. The PR branch often fails CI due to peer dep conflicts, major version breaking changes, or test infrastructure that needs the latest deps.
• Fix: Doesn't block main. The PR needs manual fixes (merge conflicts, breaking change upgrades, etc.) — or close the PR if the dep isn't critical.
• Verify: Check the main branch separately. The main branch is unaffected by Dependabot's failure.

# Detect: filter by event type and head_branch pattern
gh run list --repo camster91/<repo> --limit 20 \
  --json event,headBranch,conclusion,createdAt \
  | jq '.[] | select(.event == "pull_request" and (.headBranch | startswith("dependabot/")))'

3. cancelled — Superseded by a newer push

• Signal: conclusion: cancelled with status: completed. The run was in-flight when a newer commit was pushed.
• Why it happens: concurrency: ci-build-${{ github.ref }} + cancel-in-progress: true is the default pattern. Pushing twice in quick succession cancels the first.
• Fix: None. Look at the most recent run, not the cancelled one.
• Verify: Check the latest run on the same branch — it should be completed/success or a real failure.

4. real_source_error — Actual code problem on main

• Signal: event: push on the default branch, conclusion is failure, the error message describes a real code or config issue.
• Why it happens: Something actually broke. Common causes:
  • Lockfile not regenerated after dep change
  • prisma generate step missing from CI
  • Real source bug introduced by a commit
  • processFile used before declaration (TDZ error in TypeScript)
• Fix: Read the log, identify the file, write a targeted source fix or config update.
• Verify: Push the fix, wait 3-5 min for the next push-triggered run, confirm green.

Audit Recipe

Step 1 — Get the fleet

gh repo list --limit 200 --no-archived --json name -q . | jq -r '.[]' | sort

Filter to active repos (not archived, not deleted, not just-skeleton).

Step 2 — Per-repo, per-workflow, per-default-branch health check

For each repo, get the latest push/schedule run on the default branch. Filter out pull_request events. Filter out cancelled. Compare to the last successful push run on the same branch.

# Pseudocode
for repo in active_repos:
    branch = gh_api(f"/repos/{repo}").default_branch
    runs = gh_run_list(repo, branch=branch, limit=5)
    for run in runs:
        if run.event in ("push", "schedule"):
            if run.conclusion == "success":
                healthy.add(repo)
                break
            else:
                # Check for billing cap
                view = gh_run_view(run.id, repo)
                if "spending limit" in view:
                    billing_cap.add(repo)
                else:
                    real_issues.add((repo, run))
                break

Step 3 — Categorize the failures

Print a table:

| Category | Count | Repos |
|----------|-------|-------|
| Healthy | 39 | <list> |
| billing_cap | 16 | <list> |
| dependabot_pr | (do not count — PR noise) | <do not enumerate> |
| cancelled | (do not count — superseded) | <do not enumerate> |
| real_source_error | 1 | <list with link to log> |

Step 4 — Verify the categories

For each "real_source_error" repo, fetch the log, identify the file, and fix it. For each "billing_cap" repo, document the dependency and tell the user.

Output Format

The audit result is a Telegram-friendly markdown report:

**Fleet audit: 59 active repos**

- **39 healthy** (CI Build + Lint green on default branch)
- **16 billing cap** (waiting for user to increase GitHub Pro spending limit)
- **4 real source errors** (with files to fix)
  - `contraction-tracker`: TDZ error in `useContractions.ts` (line 47, uses `processFile` before declaration)
  - `markup-clone`: Prisma 5.22 still installed (lockfile out of sync)
  - ... etc.

The 16 billing-cap repos will all turn green after the cap resets. Nothing for me to do.

Common Pitfalls

1. Don't count Dependabot PR failures as real issues — they're a different category entirely. Reporting them as "broken" misleads the user.
2. Don't count cancelled runs as failures — they're a transient state, not a real issue. Always check the latest non-cancelled run.
3. The spending limit annotation is on the run, not the job — you need gh run view not gh run view <job-id> to see it.
4. Dependabot workflow runs in the Actions tab are confusing — there are TWO workflows: the Dependabot Updates dynamic workflow (which is fine when it succeeds) and the user's own CI Build workflow which is what's being checked. Filter on workflow name.
5. Concurrent runs create false failure summaries — the run might be cancelled at the workflow level but the conclusion is still "failure" because cancellation is a failure state from the workflow's perspective. Always check status AND conclusion.

The Two-Eye Approach

Combine the audit (programmatic, fast) with manual verification of the source errors (one at a time, with vision checks when UI is involved). Don't try to "fix all" without categorizing first — you'll waste hours debugging code that's already correct, or chasing a Dependabot branch that the user doesn't care about.

Related Skills & Chains

• eslint-flat-config-upgrade — When the audit finds repos with .eslintrc.* files, this is the next step. The audit tells you which repos need it; this skill tells you how to do the migration.
• prisma-fleet-migration — When the audit finds Prisma CLI Version 5.22.0 errors, the migration is the fix. Pair with this skill.
• dependabot-ops — When the audit finds Dependabot isn't enabled on a repo, this is how to enable it.
• spending-limit-detector — When the audit's billing-cap category is bigger than expected, this skill has more detail on detecting it.