adk-evaluation

Generate eval sets, custom metrics, simulations, and run optimization for ADK 2.0 agents

Use this skill to simulate the external environment an ADK 2.0 agent operates in — mocked APIs, fake databases, time-controlled clocks — so evals are deterministic and don't hit real services. Triggers on: "ADK environment simulation", "mock APIs for ADK eval", "ADK fake backend", "deterministic ADK testing", "ADK sandboxed eval", "stub external services ADK", "ADK eval without network". Generates fixture stubs and a test harness wiring them in place of real tool implementations.

Use this skill to generate ADK 2.0 .evalset.json files — test cases for the `adk eval` CLI. Triggers on: "ADK eval set", "ADK evalset.json", "create test cases for ADK agent", "ADK eval file", "agent test fixtures", "adk eval test data", "evaluation dataset ADK". Generates a properly structured eval set with expected tool calls, expected outputs, and metadata ready for `adk eval`.

Use this skill to run an automated optimization loop on an ADK 2.0 agent — iteratively tune instructions, models, or tool selection against an eval set until scores plateau. Triggers on: "ADK agent optimization", "auto-tune ADK agent", "improve ADK agent score", "ADK prompt optimization", "ADK iterative optimizer", "ADK eval-driven improvement", "optimize agent automatically". Generates an optimizer loop that mutates agent config, runs eval, picks the winning variant.

Use this skill to define custom evaluation metrics for ADK 2.0 agents beyond the built-in tool-match and response-similarity scores. Triggers on: "ADK custom metric", "ADK eval metric", "score ADK agent on X", "custom evaluator ADK", "ADK rubric eval", "LLM-as-judge ADK", "eval metric Python ADK". Generates a Python evaluator class (or function) that scores eval cases on domain-specific criteria.

Use this skill to run user simulations against an ADK 2.0 agent — an LLM plays the role of a user with a goal, exercising multi-turn behavior. Triggers on: "ADK user simulation", "simulate users for ADK", "ADK red team simulator", "ADK conversation tester", "synthetic user ADK", "multi-turn eval ADK", "persona-based testing ADK". Generates a user-simulator agent paired with the system-under-test plus a harness that runs N personas and scores success.