shesha-auto-testing

Imports Azure DevOps Test Plan / Test Suite test cases via the Azure DevOps MCP server (@azure-devops/mcp) and generates paired markdown test plans + recorded Playwright specs under test-plans/. Trigger phrases include "create test from devops", "import test plan from azure devops", "generate tests from test plan <id>", "pull test cases from devops", "create tests from ADO suite". The user provides an Azure DevOps Test Plan ID (and optionally a Suite ID to narrow it). The skill connects via the MCP, lists the suites and test cases, fetches each test case work item's Microsoft.VSTS.TCM.Steps field, converts the action/expected-result pairs into the prefixed TC-NN plan format per test-plans/RULES.md (one .md plan per suite, one TC per ADO test case), then records real selectors live via Playwright MCP into a paired .spec.ts using the same recording loop as the create-test skill. Reads App URL, environment, credentials, and the Azure DevOps org/project from CLAUDE.md. This skill authors only — the plans are executed later by the run-test skill, it does NOT run them.

Author a new markdown test plan at test-plans/<folder>/<name>.md AND a paired Playwright spec at test-plans/<folder>/<name>.spec.ts with real selectors recorded live via MCP browser. Trigger phrases include "create a test", "create a test plan", "new test for X", "test plan for Y", "add a test that...". The user provides only a title and a numbered list of high-level steps. The skill pulls App URL, environment, and credentials from CLAUDE.md, expands each step into full action steps per test-plans/RULES.md, matches the style of existing plans, AND walks each step against the real app via Playwright MCP to capture the resolved locator before writing the spec. Steps that can't be located after 2 retries fall back to a `// TODO[selector]:` marker for AI-repair on first run. The plan + spec are executed later by the run-test skill — this skill authors only, it does NOT run.

Dispatch a test plan to GitHub Actions instead of running it locally — the cloud-side workflow uses claude-code-action to drive the test, posts a Teams Adaptive Card on failure, and publishes an Allure report to gh-pages. Trigger phrases include "/Run-test-remote", "run test remote", "run on CI", "dispatch test on github actions", "run remote test", "trigger e2e workflow", "kick off the workflow". If `.github/workflows/e2e-test.yml` is missing the skill scaffolds it from a template and walks the user through wiring the three required secrets (`ANTHROPIC_API_KEY`, `APP_PASSWORD`, `TEAMS_WEBHOOK_URL`). If present, the skill picks plan(s) via the same disambiguation UX as `/RunTest`, dispatches each via `gh workflow run e2e-test.yml -f test_plan=<path>`, and prints the resulting run URL. Optional `--watch` flag blocks until the run completes.

Execute one or more markdown test plans from test-plans/ locally using the Playwright-first / AI-repair hybrid runner. Trigger phrases include "run test X", "run the X test", "run test-plans/<folder>/<name>.md", "execute the call log test", "run the test plan for Y". When the plan name is ambiguous (multiple matches, or none specified), the skill lists every plan with its last-run status and asks the user to pick one or more (`1,3,5`, `1-3`, or `all`). The skill invokes scripts/run-plan.js (which runs the paired .spec.ts via @playwright/test); on failure it falls back to AI-driven MCP browser execution to repair the failing line in the spec, then re-runs. After 2 failed AI-repair attempts it auto-classifies the failure by signature — stale-plan symptoms trigger a plan/spec fix, business-logic symptoms auto-log a bug under test-reports/bugs/ and the run continues. Writes a markdown report to test-reports/YYYY-MM-DD/, regenerates test-reports/index.html and the Allure report. **Does NOT push to the hub** — the user invokes /submit-test-results separately when they're ready to publish.

Self-reliant, fast bootstrap for the hybrid Markdown + Playwright testing workflow. One command — `/test-setup` — gets a fresh project to a state where `/CreateTest` works first try. Scaffolds every file the workflow needs (CLAUDE.md, package.json, playwright.config.ts, tsconfig.json, .gitignore, scripts/run-plan.js, scripts/build-dashboard.js, scripts/generate-allure-results.js, scripts/sync-to-hub.js, scripts/hub.config.example.json, test-plans/RULES.md, empty test-plans/ and test-reports/ folders) directly from templates bundled inside this skill — does NOT copy from any other project. Prompts the user once for project-specific values (project slug, app name, app URL, environment, admin credentials), substitutes them into the scaffolded files, then verifies prerequisites (Node ≥ 18, npm install, Playwright Chromium, Java for Allure) and offers to install what's missing. Registers the Playwright MCP server with Claude Code via `claude mcp add playwright -- npx -y @playwright/mcp@latest --headless` so `/CreateTest` can record selectors live without popping a visible browser window. Does NOT configure the Test-Reports-Hub or CI — those are handled by `/submit-test-results` and `/Run-test-remote` respectively when first used. Trigger phrases include "set up testing", "set up the test environment", "install test prerequisites", "check test setup", "prepare to run tests", "bootstrap testing", "scaffold testing", "initialise testing", "test-setup".