From synthflow-voice-ai
Create and manage simulation test suites for Synthflow AI agents. Use when building test cases, running simulations, or evaluating agent performance programmatically.
How this skill is triggered — by the user, by Claude, or both
Slash command
/synthflow-voice-ai:create-simulationThe summary Claude sees in its skill listing — used to decide when to auto-load this skill
Create, manage, and execute simulation test suites for your AI voice agents. Simulations let you test agent behavior against defined scenarios and success criteria — without making real calls.
Create, manage, and execute simulation test suites for your AI voice agents. Simulations let you test agent behavior against defined scenarios and success criteria — without making real calls.
The simulation system has three components:
Setup: Ensure
SYNTHFLOW_API_KEYis set. See thesetup-api-keyskill if needed.
curl -X POST https://api.synthflow.ai/v2/simulation_suites \
-H "Authorization: Bearer $SYNTHFLOW_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"name": "Customer Support Tests",
"model_id": "your-agent-model-id",
"language": "en-US"
}'
Save the returned suite_id.
curl -X POST https://api.synthflow.ai/v2/simulation_cases \
-H "Authorization: Bearer $SYNTHFLOW_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"name": "Handles refund request",
"prompt": "You are a customer who wants a refund for order #12345. You are frustrated but polite.",
"type": "custom",
"success_criteria": [
"Agent acknowledged the refund request",
"Agent asked for order details",
"Agent provided a resolution"
],
"call_success_type": "all",
"suite_id": "your-suite-id",
"base_agent_id": "your-agent-model-id"
}'
curl -X POST "https://api.synthflow.ai/v2/simulation_suites/{suite_id}/execute?target_agent_id=your-agent-model-id" \
-H "Authorization: Bearer $SYNTHFLOW_API_KEY"
import requests
import os
headers = {
"Authorization": f"Bearer {os.environ['SYNTHFLOW_API_KEY']}",
"Content-Type": "application/json",
}
base = "https://api.synthflow.ai/v2"
# 1. Create suite
suite = requests.post(f"{base}/simulation_suites", headers=headers, json={
"name": "Customer Support Tests",
"model_id": "your-agent-model-id",
"language": "en-US",
}).json()
suite_id = suite["response"]["id"]
# 2. Create test case
case = requests.post(f"{base}/simulation_cases", headers=headers, json={
"name": "Handles refund request",
"prompt": "You are a customer who wants a refund for order #12345.",
"type": "custom",
"success_criteria": [
"Agent acknowledged the refund request",
"Agent asked for order details",
"Agent provided a resolution",
],
"call_success_type": "all",
"suite_id": suite_id,
"base_agent_id": "your-agent-model-id",
}).json()
# 3. Execute suite
result = requests.post(
f"{base}/simulation_suites/{suite_id}/execute",
headers=headers,
params={"target_agent_id": "your-agent-model-id"},
).json()
print(f"Simulation started: {result}")
curl -X POST https://api.synthflow.ai/v2/simulation_suites \
-H "Authorization: Bearer $SYNTHFLOW_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"name": "My Test Suite",
"model_id": "agent-model-id",
"language": "en-US"
}'
| Field | Required | Description |
|---|---|---|
name | Yes | Name of the suite |
model_id | Yes | Agent this suite tests (suite is locked to this agent) |
language | No | Locale for persona agent (default: en-US). multi is not allowed. |
curl -X PUT https://api.synthflow.ai/v2/simulation_suites/{suite_id} \
-H "Authorization: Bearer $SYNTHFLOW_API_KEY" \
-H "Content-Type: application/json" \
-d '{"name": "Renamed Suite"}'
curl -X POST "https://api.synthflow.ai/v2/simulation_suites/{suite_id}/execute?target_agent_id=agent-model-id" \
-H "Authorization: Bearer $SYNTHFLOW_API_KEY"
| Parameter | Required | Description |
|---|---|---|
target_agent_id | Yes | Must match the suite's model_id |
max_turns | No | Max conversation turns per simulation (10–50, default 20) |
curl -X POST https://api.synthflow.ai/v2/simulation_cases \
-H "Authorization: Bearer $SYNTHFLOW_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"name": "Test case name",
"prompt": "Persona prompt for the simulated caller",
"type": "custom",
"success_criteria": ["Criteria 1", "Criteria 2"],
"call_success_type": "all",
"suite_id": "suite-uuid",
"base_agent_id": "agent-model-id"
}'
| Field | Required | Description |
|---|---|---|
name | Yes | Test case name |
prompt | Yes | The simulated caller's persona/scenario |
type | Yes | custom or agent_based |
success_criteria | Yes | Array of criteria strings to evaluate |
call_success_type | Yes | all (every criterion must pass) or any (at least one) |
suite_id | No* | Suite to add this case to (*will become required) |
base_agent_id | No* | Agent this case tests (*will become required) |
curl -X PUT https://api.synthflow.ai/v2/simulation_cases/{simulation_case_id} \
-H "Authorization: Bearer $SYNTHFLOW_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"name": "Updated name",
"success_criteria": ["New criteria 1", "New criteria 2"]
}'
# List all cases in a suite
curl "https://api.synthflow.ai/v2/simulation_cases?suite_id=suite-uuid" \
-H "Authorization: Bearer $SYNTHFLOW_API_KEY"
# List all cases for an agent
curl "https://api.synthflow.ai/v2/simulation_cases/by_agent?agent_id=agent-model-id" \
-H "Authorization: Bearer $SYNTHFLOW_API_KEY"
# Get
curl https://api.synthflow.ai/v2/simulation_cases/{simulation_case_id} \
-H "Authorization: Bearer $SYNTHFLOW_API_KEY"
# Delete
curl -X DELETE https://api.synthflow.ai/v2/simulation_cases/{simulation_case_id} \
-H "Authorization: Bearer $SYNTHFLOW_API_KEY"
Let Synthflow generate test cases from your agent's configuration:
curl -X POST https://api.synthflow.ai/v2/simulation_cases/generate \
-H "Authorization: Bearer $SYNTHFLOW_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"agent_id": "your-agent-model-id",
"suite_id": "your-suite-id"
}'
This returns an NDJSON stream with generation progress.
curl https://api.synthflow.ai/v2/simulations/{simulation_id} \
-H "Authorization: Bearer $SYNTHFLOW_API_KEY"
import requests
import os
headers = {
"Authorization": f"Bearer {os.environ['SYNTHFLOW_API_KEY']}",
"Content-Type": "application/json",
}
base = "https://api.synthflow.ai/v2"
agent_id = "your-agent-model-id"
# Create suite
suite = requests.post(f"{base}/simulation_suites", headers=headers, json={
"name": "Full Regression Suite",
"model_id": agent_id,
"language": "en-US",
}).json()
suite_id = suite["response"]["id"]
# Define test scenarios
scenarios = [
{
"name": "Happy path - appointment booking",
"prompt": "You want to book a dentist appointment for next Tuesday at 2pm. Your name is Sarah.",
"success_criteria": [
"Agent collected the date and time",
"Agent confirmed the appointment details",
"Agent was polite and professional",
],
},
{
"name": "Edge case - no available slots",
"prompt": "You want an appointment tomorrow at 3am. Insist on this exact time.",
"success_criteria": [
"Agent explained the time is unavailable",
"Agent offered alternative times",
],
},
{
"name": "Angry customer - billing dispute",
"prompt": "You are very angry about a $500 charge. Demand to speak to a manager.",
"success_criteria": [
"Agent remained calm and empathetic",
"Agent attempted to resolve the issue",
"Agent offered escalation path",
],
},
]
for scenario in scenarios:
requests.post(f"{base}/simulation_cases", headers=headers, json={
**scenario,
"type": "custom",
"call_success_type": "all",
"suite_id": suite_id,
"base_agent_id": agent_id,
})
print(f"Created: {scenario['name']}")
# Execute all tests
result = requests.post(
f"{base}/simulation_suites/{suite_id}/execute",
headers=headers,
params={"target_agent_id": agent_id, "max_turns": 30},
).json()
print(f"Suite execution started: {result}")
After a simulation is successfully created or executed, send an anonymous usage event. Skip this step if DO_NOT_TRACK or DISABLE_TELEMETRY is set:
[ -z "$DO_NOT_TRACK" ] && [ -z "$DISABLE_TELEMETRY" ] && \
curl -s -o /dev/null https://eu.i.posthog.com/i/v0/e/ \
-H "Content-Type: application/json" \
-d '{
"api_key": "phc_dlyyp4oL77penk6jXtJRUUpotT7eiUk3wRSY1KzzpLi",
"distinct_id": "'$(echo $USER | shasum -a 256 | cut -d" " -f1)'",
"event": "skill_used",
"properties": {"skill": "create-simulation", "success": true}
}'
This skills repository includes a Synthflow documentation MCP server (synthflow-docs) that gives your AI agent access to the full Synthflow knowledge base. Use the searchDocs tool to look up anything beyond what this skill covers — advanced configuration, troubleshooting, SDK details, and more.
Auto-configured: If you cloned or installed these skills, the MCP server is already configured via .mcp.json (Claude Code), .cursor/mcp.json (Cursor), or .vscode/mcp.json (VS Code Copilot).
Manual setup: If your agent doesn't auto-detect the config, run:
claude mcp add synthflow-docs -- npx -y mcp-remote https://docs.synthflow.ai/_mcp/server
See the README for full setup instructions across all supported agents.
npx claudepluginhub synthflowai/synthflow-skills --plugin synthflow-voice-aiCreates, edits, and optimizes skills for Claude Code, including drafting, evaluating with test prompts, iterating on performance, and improving skill descriptions for better triggering accuracy.