From aws-test-plugin
Master skill for testing AWS Python projects. Discovers project components (Lambda, API Gateway, Step Functions, Batch), reads handler code logic, and coordinates test generation across all categories (E2E, integration, contract, performance, load). Use when asked to: test, write tests, scaffold tests, run tests, analyze test results, debug test failures, or improve test coverage for any AWS Python microservice project.
How this skill is triggered — by the user, by Claude, or both
Slash command
/aws-test-plugin:aws-test-orchestratorThe summary Claude sees in its skill listing — used to decide when to auto-load this skill
Master coordination skill. Discovers the project, delegates to category-specific skills.
Master coordination skill. Discovers the project, delegates to category-specific skills.
Before writing any test, map the project. Run these discovery steps.
Every component gets a unique env var prefix to avoid collisions in monorepos. Derive it from the service directory name, uppercased with hyphens replaced by underscores:
| Directory | Prefix | Example Env Vars |
|---|---|---|
orders-api/ | ORDERS_API | ORDERS_API_BASE_URL, ORDERS_API_KEY |
processing-batch/ | PROCESSING_BATCH | PROCESSING_BATCH_BATCH_JOB_QUEUE |
order-processor/ | ORDER_PROCESSOR | ORDER_PROCESSOR_SFN_ARN |
notifications/ | NOTIFICATIONS | NOTIFICATIONS_QUEUE_URL |
Use this prefix consistently in .env.example, conftest.py fixtures, and
E2E test modules. See references/secrets-and-config.md.
Search for files named main.py, handler.py, app.py, or lambda_function.py
under directories that contain lambda, function, or handler in their path.
Each directory containing such a file is one Lambda function.
Record for each Lambda:
def lambda_handler, def handler, or the entry point)httpMethod/requestContext), S3 (has Records[].s3), SQS (has Records[].body), SNS, EventBridge, direct invokeboto3.client(...) or boto3.resource(...) — record each service name (s3, dynamodb, rds, stepfunctions, etc.)os.environ[...] or os.getenv(...) callsSearch for directories containing Dockerfile, batch, or job in their name.
Look for the main entry point (main.py, run.py, app.py).
Record for each Batch job:
Search for files named *.asl.json, state_machine.json, definition.json,
or CloudFormation templates that define AWS::StepFunctions::StateMachine.
Record for each Step Function:
Search for swagger.yml, swagger.yaml, openapi.yml, openapi.yaml, openapi.json,
or API Gateway definitions in CloudFormation/SAM templates.
Record:
Search for existing test files: test_*.py, *_test.py
Also look for conftest.py, pytest.ini, pyproject.toml [tool.pytest], setup.cfg [tool:pytest]
Record:
Based on what the user asked, delegate to the appropriate skill:
| User Says | Skill | What It Does |
|---|---|---|
| "unit test", "test business logic", "branch coverage", "boundary test" | aws-unit-testing | Tests pure logic, every branch and boundary |
| "E2E test", "end-to-end", "full workflow", "test the API" | aws-e2e-testing | Tests real deployed endpoints |
| "integration test", "test with S3", "test with DynamoDB" | aws-integration-testing | Tests service + mocked AWS |
| "contract test", "schema test", "API contract" | aws-contract-testing | Validates schemas offline |
| "performance", "benchmark", "latency", "memory" | aws-perf-load-testing | Benchmarks and profiling |
| "load test", "stress test", "concurrent users" | aws-perf-load-testing | Locust-based load testing |
| "all tests", "full test suite", "scaffold tests" | All skills | Run discovery → scaffold → generate |
Before generating any tests, ensure secrets management is set up correctly. See references/secrets-and-config.md for full details.
Key rules:
.env files (gitignored) + os.getenv() — quick local dev~/.aws/credentials) — accessed via boto3 automatically"testing" via moto).env or Secrets Manager / SSM with pytest.skip() fallback.env.example (committed) and .gitignore entry for .envThis is the core differentiator. Do NOT use generic templates. Instead:
boto3 call → what service, method, and expected parameters?test_<function>_<scenario>_<expected_outcome>def test_handler_valid_get_request_returns_200():
def test_handler_missing_path_param_returns_400():
def test_handler_s3_object_not_found_returns_404():
def test_handler_dynamodb_condition_check_fails_returns_409():
def test_handler_unexpected_error_returns_500():
If the project doesn't have a test directory, create one:
tests/
├── __init__.py
├── conftest.py # Shared fixtures (auto-generated from discovery)
├── pytest.ini # Markers: unit, contract, integration, e2e, performance, load
├── unit/ # Pure business logic tests
│ └── __init__.py
├── contract/
│ └── __init__.py
├── integration/
│ └── __init__.py
├── e2e/
│ └── __init__.py
├── performance/
│ └── __init__.py
├── load/
│ └── __init__.py
└── reports/ # Output directory
.env.example # Non-secret config template (committed)
.env # Non-secret config values (gitignored, NEVER committed)
Generate conftest.py from discovery results:
# Auto-generated fixtures based on project discovery
import os
import pytest
from pathlib import Path
from unittest.mock import MagicMock
REPO_ROOT = Path(__file__).resolve().parent.parent
# --- Environment fixtures ---
@pytest.fixture(autouse=True)
def clean_env(monkeypatch):
"""Reset env vars between tests."""
# Set each env var discovered in Step 1
for key, default in DISCOVERED_ENV_VARS.items():
monkeypatch.setenv(key, default)
# --- AWS credential fixtures ---
@pytest.fixture
def aws_credentials(monkeypatch):
"""Mock AWS credentials for moto."""
monkeypatch.setenv("AWS_ACCESS_KEY_ID", "testing")
monkeypatch.setenv("AWS_SECRET_ACCESS_KEY", "testing")
monkeypatch.setenv("AWS_SECURITY_TOKEN", "testing")
monkeypatch.setenv("AWS_SESSION_TOKEN", "testing")
monkeypatch.setenv("AWS_DEFAULT_REGION", "us-east-1")
# --- Lambda context fixture ---
@pytest.fixture
def mock_lambda_context():
ctx = MagicMock()
ctx.function_name = "test-function"
ctx.memory_limit_in_mb = 256
ctx.invoked_function_arn = "arn:aws:lambda:us-east-1:123456789:function:test"
ctx.get_remaining_time_in_millis.return_value = 30000
return ctx
# --- API spec fixture (if OpenAPI found) ---
@pytest.fixture(scope="session")
def api_spec():
spec_path = REPO_ROOT / "PATH_TO_SPEC" # Replace with discovered path
if spec_path.exists():
import yaml
return yaml.safe_load(spec_path.read_text(encoding="utf-8"))
return None
@pytest.fixture(scope="session")
def api_schemas(api_spec):
if api_spec:
return api_spec.get("components", {}).get("schemas", {})
return {}
| Category | Command |
|---|---|
| Unit | pytest tests/unit/ -v -m unit --cov=src/ --cov-branch |
| Contract | pytest tests/contract/ -v -m contract --tb=short |
| Integration | pytest tests/integration/ -v -m integration --tb=short |
| E2E | pytest tests/e2e/ -v -m e2e -s (loads secrets from .env or SM/SSM) |
| Performance | pytest tests/performance/ -v --benchmark-only |
| Load | locust -f tests/load/locustfile.py --host=$LOAD_TEST_HOST --users=50 --run-time=5m --headless |
| All offline | pytest tests/unit/ tests/contract/ tests/integration/ tests/performance/ -v |
| With reports | pytest tests/ -v --junitxml=tests/reports/junit.xml --html=tests/reports/report.html --self-contained-html |
python scripts/run_tests.py unit # Unit tests only
python scripts/run_tests.py contract # Run one category
python scripts/run_tests.py all # All offline categories (unit+contract+integration+performance)
python scripts/run_tests.py e2e # E2E (tests self-skip if per-component env vars missing)
python scripts/run_tests.py load --users 100 # Load test (requires --base-url or LOAD_TEST_HOST)
python scripts/run_tests.py full # Everything: offline + E2E (tests self-skip if secrets unavailable)
python scripts/analyze_results.py # All reports
python scripts/analyze_results.py --json # Machine-readable
| Symptom | Root Cause | Fix |
|---|---|---|
ValidationError in contract test | Schema mismatch | Update OpenAPI spec or handler code |
ClientError in integration test | Wrong boto3 call | Fix service name, method, or params |
AssertionError on response body | Handler logic bug | Fix the handler's data transformation |
TimeoutError in E2E | Slow downstream | Check Lambda timeout, DB query, SFN wait |
ConnectionError in E2E | Wrong URL or no VPN | Verify API_BASE_URL and network access |
| Benchmark regression | Code got slower | Profile with cProfile/tracemalloc |
| Load test error rate > 1% | Throttling or crashes | Check CloudWatch, scale up, optimize |
1. RUN → Execute failing test(s) only
2. READ → Parse the failure traceback
3. TRACE → Find the failing line in handler code
4. FIX → Apply minimal fix
5. RE-RUN → Run ONLY the previously failing test
6. REPEAT → Until green
7. FULL → Run full category suite to check for regressions
E2E-TEST-, LOAD-TEST-)--run-time set (no unbounded runs).env or Secrets Manager / SSM for real secretsnpx claudepluginhub whitewhiteqq/aws-test-plugin --plugin claude-aws-test-pluginWrites and runs unit, integration, e2e, performance, and contract tests to verify code functionality.
Generates and runs unit, integration (testcontainers/docker-compose), and Playwright E2E test suites for JS/TS code. Analyzes coverage gaps with parallel test-generator agents, executes tests, and heals failures up to 3 times.
Build API test suites — endpoint testing, contract testing, load testing for REST/GraphQL/gRPC APIs. Use when asked to "test this API", "API tests", "endpoint testing", "contract tests", or "load test".