Test Plan
End-to-end test planning workflow for RHOAI: generate test plans from strategies, create test cases, implement executable automation code, verify UI tests against live clusters via Playwright, publish to GitHub with PR creation, resolve review feedback, and score quality with automated rubrics using parallel sub-agent analysis.
Skills
User-invocable (8 skills)
| Skill | Description |
|---|
/test-plan-create | Generate a test plan from a strategy (RHAISTRAT or RHOAIENG), with optional ADR |
/test-plan-create-cases | Generate individual test case files from an existing test plan |
/test-plan-update | Update test plan with new docs (ADR, API specs), re-analyze, bump version |
/test-plan-case-implement | Generate executable test automation code from TC specifications with intelligent placement |
/test-plan-ui-verify | Verify UI test cases from a PR against a live ODH/RHOAI cluster via Playwright |
/test-plan-publish | Publish test plan artifacts to GitHub — branch, commit, and open a PR |
/test-plan-resolve-feedback | Assess PR review comments, let the user decide what to apply, and push updates |
/test-plan-score | Score an existing test plan using quality rubric (without auto-revision) |
Sub-agents (9 internal skills, forked with context: fork)
| Skill | Description |
|---|
test-plan-analyze-endpoints | Extract feature scope, test objectives, and API endpoints from docs |
test-plan-analyze-risks | Analyze strategy/ADR to determine test levels, types, priorities, risks |
test-plan-analyze-infra | Identify test environment, data, infrastructure requirements |
test-plan-analyze-placement | Recommend test placement (component repo vs downstream) |
test-plan-merge | Intelligently merge new analyzer findings into existing test plan |
test-plan-resolve-gaps | Cross-reference gaps with new findings to determine what's resolved |
test-plan-review | Review test plan for completeness, consistency, and quality with auto-revision |
test-plan-generate-test-file | Generate complete test file with all functions, quality scoring and auto-revision |
test-plan-score-test-function | Score generated test function code using 5-criteria quality rubric |
Installation
Option 1: Install from Marketplace (Recommended)
Install from the opendatahub-io/skills-registry marketplace:
# Add the marketplace (one-time)
claude plugin marketplace add opendatahub-io/skills-registry
# Install test-plan plugin
/plugin install test-plan@opendatahub-skills
This clones the repository and makes skills immediately available. Then install Python dependencies:
cd ~/.claude/plugins/test-plan
uv sync --extra dev
Use skills:
# Will prompt for artifact location (default: ~/Code/opendatahub-test-plans)
/test-plan-create RHAISTRAT-400
# Auto-uses location from /test-plan-create
/test-plan-create-cases
# Auto-detects feature from /test-plan-create session
# Will prompt for publish target (default: opendatahub-io/opendatahub-test-plans)
/test-plan-publish
Option 2: Manual Clone (For Contributors)
Clone the repository directly:
git clone https://github.com/opendatahub-io/odh-test-gen ~/Code/odh-test-gen
cd ~/Code/odh-test-gen
uv sync --extra dev
Skills are available from skills/ directory.
Note: Skills use symlinks for shared utilities (_common/scripts → ../../scripts). Both installation methods clone the full repository, so symlinks resolve correctly.
Each skill includes an argument-hint field in its frontmatter for autocomplete guidance when typing slash commands.
Artifact Location
Important: Test plan artifacts are kept separate from the skill repository to maintain clean boundaries between code and data.
Default Behavior
When you run /test-plan-create, it asks where to save artifacts:
Where should test plan artifacts be created?
Provide a directory path (e.g., ~/Code/opendatahub-test-plans/plans/<team-name>), or press Enter for: ~/Code/opendatahub-test-plans/plans/
Note: Replace <team-name> with your team name (e.g., ai-hub, dashboard, etc.)
- QE Teams: Provide your team's directory path (e.g.,
~/Code/opendatahub-test-plans/plans/ai-hub)
- Contributors: Use default
~/Code/opendatahub-test-plans/plans/ or any path outside the skill repo
- Save Preference: Optionally save your choice to
.claude/settings.json for future runs
The skill creates: <your-path>/<feature-name>/
Example: If you enter ~/Code/opendatahub-test-plans/plans/ai-hub, it creates ~/Code/opendatahub-test-plans/plans/ai-hub/mcp_catalog/
Session Context
/test-plan-create-cases automatically uses the same location as /test-plan-create when run in the same session (no prompt needed).
Publishing
