From kibana-testing-tools
Auto-generate comprehensive Scout API tests from Kibana route definitions, including valid/invalid request tests, RBAC tests, and schema-driven test data generation.
How this skill is triggered — by the user, by Claude, or both
Slash command
/kibana-testing-tools:api-test-generatorThe summary Claude sees in its skill listing — used to decide when to auto-load this skill
Auto-generate comprehensive Scout API tests from Kibana route definitions.
Auto-generate comprehensive Scout API tests from Kibana route definitions.
Parse versioned Kibana route definitions and generate complete test suites including:
Use this skill when:
First, identify the route file. Ask the user for the file path if not provided.
Expected patterns:
// Versioned routes
router.versioned
.post({
path: '/api/plugin_name/endpoint',
access: 'public' | 'internal',
security: {
authz: {
requiredPrivileges: ['cluster_monitor', 'manage_index_templates'],
},
},
})
.addVersion({
version: '2023-10-31',
validate: {
request: {
body: schema.object({
name: schema.string(),
value: schema.number(),
}),
query: schema.object({
filter: schema.maybe(schema.string()),
}),
params: schema.object({
id: schema.string(),
}),
},
response: {
200: {
body: () => schema.object({
id: schema.string(),
status: schema.string(),
}),
},
},
},
});
Extract:
security.authz.requiredPrivilegesMap requiredPrivileges to user roles:
Privilege → Role mapping:
const PRIVILEGE_ROLE_MAP = {
// Monitoring
'cluster_monitor': ['viewer', 'editor', 'admin'],
'monitor': ['viewer', 'editor', 'admin'],
// Management
'manage_index_templates': ['editor', 'admin'],
'manage_ingest_pipelines': ['editor', 'admin'],
'manage': ['admin'],
// Write operations
'write': ['editor', 'admin'],
'create': ['editor', 'admin'],
// Read operations
'read': ['viewer', 'editor', 'admin'],
'view_index_metadata': ['viewer', 'editor', 'admin'],
// All
'all': ['admin'],
};
Determine:
For each schema field, generate:
Valid data:
// From schema.string()
validString: 'test-value'
// From schema.string({ minLength: 5, maxLength: 50 })
validString: 'valid-test-string'
// From schema.number({ min: 0, max: 100 })
validNumber: 42
// From schema.boolean()
validBoolean: true
// From schema.arrayOf(schema.string())
validArray: ['item1', 'item2']
// From schema.object()
validObject: { nested: 'value' }
// From schema.maybe() or schema.nullable()
optionalField: undefined // or valid value
Invalid data:
// Wrong type
invalidString: 123
invalidNumber: 'not-a-number'
invalidBoolean: 'not-a-boolean'
// Violates constraints
tooShort: 'abc' // when minLength: 5
tooLong: 'x'.repeat(100) // when maxLength: 50
outOfRange: -10 // when min: 0
// Missing required field
missingRequired: { /* omit required field */ }
// Extra fields
extraFields: { validField: 'value', unexpected: 'field' }
Use this template structure:
import { apiTest, expect } from '@kbn/scout/api';
import { COMMON_HEADERS } from '../constants';
apiTest.describe('[METHOD] [PATH]', () => {
// ============================================================================
// Valid Request Tests
// ============================================================================
apiTest('returns 200 with valid request (admin)', async ({ apiClient, requestAuth, log }) => {
const adminCreds = await requestAuth.getApiKeyForAdmin();
const response = await apiClient.[method]('[path]', {
headers: { ...COMMON_HEADERS, ...adminCreds.apiKeyHeader },
body: {
// Valid body data
},
query: {
// Valid query params
},
params: {
// Valid path params (if path has :id style params)
},
});
expect(response).toHaveStatusCode(200);
expect(response.body).toMatchObject({
// Expected response shape
});
log.info('Response:', response.body);
});
// ============================================================================
// RBAC Tests
// ============================================================================
// For each role that SHOULD succeed
apiTest('returns 200 for [role] (has required privileges)', async ({ apiClient, requestAuth }) => {
const creds = await requestAuth.getApiKeyFor[Role](); // getApiKeyForEditor, getApiKeyForViewer
const response = await apiClient.[method]('[path]', {
headers: { ...COMMON_HEADERS, ...creds.apiKeyHeader },
body: { /* valid data */ },
});
expect(response).toHaveStatusCode(200);
});
// For each role that SHOULD fail
apiTest('returns 403 for [role] (missing required privileges)', async ({ apiClient, requestAuth }) => {
const creds = await requestAuth.getApiKeyFor[Role]();
const response = await apiClient.[method]('[path]', {
headers: { ...COMMON_HEADERS, ...creds.apiKeyHeader },
body: { /* valid data */ },
});
expect(response).toHaveStatusCode(403);
expect(response.body).toHaveProperty('message');
});
// ============================================================================
// Authentication Tests
// ============================================================================
apiTest('returns 401 without authentication', async ({ apiClient }) => {
const response = await apiClient.[method]('[path]', {
headers: COMMON_HEADERS, // No auth headers
body: { /* valid data */ },
});
expect(response).toHaveStatusCode(401);
});
// ============================================================================
// Validation Tests
// ============================================================================
apiTest('returns 400 with invalid request body', async ({ apiClient, requestAuth }) => {
const adminCreds = await requestAuth.getApiKeyForAdmin();
const response = await apiClient.[method]('[path]', {
headers: { ...COMMON_HEADERS, ...adminCreds.apiKeyHeader },
body: {
// Invalid data (wrong type, missing field, constraint violation)
},
});
expect(response).toHaveStatusCode(400);
expect(response.body).toHaveProperty('message');
expect(response.body.message).toContain('validation'); // or specific error
});
apiTest('returns 400 with missing required fields', async ({ apiClient, requestAuth }) => {
const adminCreds = await requestAuth.getApiKeyForAdmin();
const response = await apiClient.[method]('[path]', {
headers: { ...COMMON_HEADERS, ...adminCreds.apiKeyHeader },
body: {
// Omit required field
},
});
expect(response).toHaveStatusCode(400);
});
// ============================================================================
// Edge Case Tests
// ============================================================================
apiTest('handles empty strings', async ({ apiClient, requestAuth }) => {
const adminCreds = await requestAuth.getApiKeyForAdmin();
const response = await apiClient.[method]('[path]', {
headers: { ...COMMON_HEADERS, ...adminCreds.apiKeyHeader },
body: {
stringField: '',
},
});
// Expect 400 if empty not allowed, or 200 if allowed
expect(response).toHaveStatusCode([expectedStatus]);
});
apiTest('handles maximum length strings', async ({ apiClient, requestAuth }) => {
const adminCreds = await requestAuth.getApiKeyForAdmin();
const response = await apiClient.[method]('[path]', {
headers: { ...COMMON_HEADERS, ...adminCreds.apiKeyHeader },
body: {
stringField: 'x'.repeat([maxLength]),
},
});
expect(response).toHaveStatusCode(200);
});
apiTest('rejects strings exceeding maximum length', async ({ apiClient, requestAuth }) => {
const adminCreds = await requestAuth.getApiKeyForAdmin();
const response = await apiClient.[method]('[path]', {
headers: { ...COMMON_HEADERS, ...adminCreds.apiKeyHeader },
body: {
stringField: 'x'.repeat([maxLength + 1]),
},
});
expect(response).toHaveStatusCode(400);
});
// Add more edge cases based on schema (special chars, unicode, boundary values)
});
Generate:
Test file path suggestion:
x-pack/test/api_integration/apis/[plugin_name]/[endpoint_name].scout.ts
Full test file content (ready to copy-paste)
Test constants file (if needed):
// x-pack/test/api_integration/apis/[plugin_name]/constants.ts
export const COMMON_HEADERS = {
'kbn-xsrf': 'kibana',
'Content-Type': 'application/json',
};
Scout config update (if new test area):
// x-pack/test/api_integration/configs/[plugin_name].scout.ts
import { ScoutServerConfig } from '@kbn/scout';
export const config: ScoutServerConfig = {
serverless: false,
projectType: 'es',
services: {},
};
Handle versioned paths:
// If path contains version in URL: /api/v1/endpoint
// Use the path as-is
// If route uses .addVersion(), note the version in test description
apiTest.describe('POST /api/endpoint [v2023-10-31]', () => { ... });
Handle parameterized paths:
// For paths like: '/api/cases/{case_id}/comments'
// Use concrete value in test:
const response = await apiClient.post('api/cases/test-case-123/comments', {
params: { case_id: 'test-case-123' },
// ...
});
Handle optional auth:
// If route has access: 'public' and no requiredPrivileges
// Test both authenticated and unauthenticated access
apiTest('works without authentication (public endpoint)', async ({ apiClient }) => {
const response = await apiClient.get('[path]', {
headers: COMMON_HEADERS,
});
expect(response).toHaveStatusCode(200);
});
After generating tests:
Ask for clarification when:
Provide:
node scripts/scout run-tests --arch stateful --domain classic --testFiles x-pack/test/api_integration/apis/[plugin]/[endpoint].scout.ts
User: "Generate API tests for the alerts API route in x-pack/plugins/alerting/server/routes/create_rule.ts"
You:
x-pack/test/api_integration/apis/alerting/create_rule.scout.tsapiTest from @kbn/scout/api, not test from PlaywrightrequestAuth.getApiKeyForAdmin(), getApiKeyForEditor(), getApiKeyForViewer()log.info() in success cases for debuggingnpx claudepluginhub patrykkopycinski/patryks-treadmill-claude-plugins --plugin kibana-testing-toolsGuides creation, editing, and verification of skills for AI coding agents using test-driven development with subagent scenarios. Use when authoring or debugging skills.