Stats
Actions
Available In
Tags
Plugin
qa-ml-models
ML model testing: 5 skills (giskard-tests, deepchecks-tests, evidently-monitoring, fairlearn-fairness, alibi-explainability) and 1 agent (model-fairness-reviewer). Covers tabular + NLP + vision validation, drift monitoring, group fairness, and per-prediction explainability.
What's Inside
Agents2
data-drift-incident-responder
/data-drift-incident-responder
Receives a live Evidently drift alert (HTML or JSON report) and produces a ranked root-cause hypothesis list plus a remediation checklist. Distinguishes upstream schema change, seasonality, training-serving skew, pipeline bug, and genuine population shift; recommends rollback, retrain, quarantine, feature investigation, or alert re-tuning as appropriate. Use when a DataDriftPreset or TestColumnDrift alert fires in production monitoring and the on-call engineer needs a structured triage before acting.
model-fairness-reviewer
/model-fairness-reviewer
Adversarial reviewer of ML model fairness + explainability evidence before promotion. Validates that fairness metrics (Fairlearn MetricFrame), drift detectors (Evidently/Deepchecks), vulnerability scans (Giskard), and per-prediction explanations (Alibi) collectively cover the model's risk class. Refuses to ✅ when sensitive features are missing, when intersectional analysis is absent, or when a high-risk model lacks per-prediction explanation logging.
Skills6
alibi-explainability
/alibi-explainability
Use Alibi Explain to generate model explanations - Anchors, Integrated Gradients, Kernel/Tree SHAP, ALE, Counterfactual Instances. Wires explainer.fit + explainer.explain into model-evaluation pipelines so that every flagged prediction ships with a "why" record auditors can reason about.
deepchecks-tests
/deepchecks-tests
Run Deepchecks suites (data integrity, train-test validation, model evaluation) on tabular / NLP / vision data + models. Pass `result.passed_conditions()` to CI to gate on regressions; the same checks run during research, CI, and production monitoring per the Deepchecks lifecycle posture.
evidently-monitoring
/evidently-monitoring
Use Evidently OSS (100+ evaluation metrics, declarative testing API) to detect data drift, target drift, and model performance regression - wired into CI as a gate and into production monitoring as a continuous check. Reports as HTML + JSON for both human review and pipeline assertions.
fairlearn-fairness
/fairlearn-fairness
Compute group fairness metrics (selection rate, demographic parity, equalized odds) per sensitive feature with `MetricFrame`, then mitigate disparities using Reductions algorithms (`ExponentiatedGradient` with constraint = `DemographicParity`/`EqualizedOdds`). Wire group-disaggregated assertions into the model-evaluation gate.
giskard-tests
/giskard-tests
Test ML models with Giskard's scan() vulnerability detector + test catalog (performance, robustness, fairness, data leakage, ethical issues) for tabular and NLP models. Wrap a prediction function in giskard.Model + a DataFrame in giskard.Dataset; emit test suites that pass/fail in CI.
Stats
Version1.2.0
LanguagePython
Stars0
MaintenanceExcellent
LicenseMIT
Last Commit
Added
Available In
Safety Signals
Caution
Uses power tools
Uses Bash, Write, or Edit tools
Tags
Categories
