mltoolkit

mltoolkit — Claude Code ML Plugin

A paper-grade machine-learning plugin for Claude Code. Generates native Python (scikit-learn + optional XGBoost/LightGBM/CatBoost/Optuna/SHAP) for classification, regression, clustering, and anomaly detection — no PyCaret dependency — with TRIPOD+AI / STARD / CONSORT-AI reporting scaffolds built in.

Every run emits: leaderboard, per-fold scores, calibration, bootstrap CIs, subgroup metrics, fairness disparities, decision-curve analysis, reliability diagram, SHAP, learning curve, Table 1, EPV audit, datasheet.md, methods.md, model_card.md, TRIPOD+AI checklist, run_manifest.json, and a packaged deliverable you can drop into a repo.

---

Install

Option A — Claude Code plugin marketplace (recommended)

# Inside a Claude Code session:
/plugin marketplace add olaTechie/mltoolkit-plugin
/plugin install mltoolkit@olaTechie

After install, verify the skills are registered:

/plugin list

You should see mltoolkit:setup, mltoolkit:classify, mltoolkit:regress, mltoolkit:cluster, mltoolkit:anomaly, mltoolkit:compare, mltoolkit:tune, mltoolkit:eda, mltoolkit:package.

Option B — Local clone (for development / forking)

git clone https://github.com/olaTechie/mltoolkit-plugin.git
cd mltoolkit-plugin
claude --plugin-dir .

Option C — Per-repo pin

Inside a project that will use the plugin, create .claude/plugins.json:

{
  "plugins": [
    {
      "name": "mltoolkit",
      "source": "github:olaTechie/mltoolkit-plugin"
    }
  ]
}

Verify

bash scripts/check-env.sh
bash tests/test_references.sh   # optional: 67-test smoke suite

---

Requirements

Required: Python ≥ 3.9, pandas, numpy, scikit-learn, scipy, matplotlib, seaborn, joblib.

Optional (additive features when installed):

Package	Unlocks
xgboost, lightgbm, catboost	Extra models in the classify/regress zoos
imbalanced-learn	--resample {smote,adasyn}
category_encoders	TargetEncoder for non-sensitive high-cardinality columns
optuna	--search-library optuna (TPE sampler)
shap	SHAP beeswarm plot in evaluate
mlflow	--track mlflow experiment logging
pyod	anomaly zoo: abod, hbos, cof, sod, sos
kmodes	cluster zoo: kmodes

All optional deps are gracefully skipped when absent (the plugin prints a warning and falls back).

---

Skills at a glance

Skill	Purpose	Primary outputs
mltoolkit:setup	Load data, EDA, task detection, ethics datasheet	schema.csv, datasheet.md, correlation_heatmap.png
mltoolkit:classify	Binary/multiclass classification (full paper-mode)	leaderboard, calibration, subgroup, SHAP, reports/
mltoolkit:regress	Regression with robust estimators + skew-aware CV	leaderboard, residuals, Q-Q, bootstrap CIs
mltoolkit:cluster	KMeans/DBSCAN/Agglom/GMM/AP/MeanShift/Spectral/OPTICS/Birch	leaderboard, elbow, PCA scatter, assigned.csv
mltoolkit:anomaly	iForest/LOF/Elliptic/OCSVM/PCA/MCD (+pyod)	scores.csv, top_anomalies.csv, subgroup rates
mltoolkit:compare	Re-run model comparison with new flags	leaderboard + per-fold
mltoolkit:tune	Hyperparameter search (sklearn or optuna)	best_params.json
mltoolkit:eda	Regenerate EDA figures (Table 1, missingness, EPV)	table1.csv, epv_audit.json
mltoolkit:package	Tier A (single file) / B (mini project) / C (full scaffold)	deliverable + pinned requirements + reports

---

Sample prompts

Copy-paste any of these at the Claude Code prompt. Claude will invoke the right skill and generate native Python in your CWD.

Quickstart — binary classifier on a CSV

Use mltoolkit:setup on data/diabetes.csv with target "outcome".
Then classify it and package the result as a mini project called "diabetes_model".

Paper-grade clinical-prediction run

I have a cohort at data/patients.csv with target "readmitted_30d".
The columns "race", "sex", and "zip_code" are protected attributes.
I want:
  - group-fairness metrics by race
  - calibration + reliability diagram
  - 95% bootstrap CIs on holdout
  - decision-curve analysis
  - TRIPOD+AI reporting scaffold
  - finalized model refit on the full dataset

Use mltoolkit:classify.

Claude will generate a staged .mltoolkit/session.py and run it with:

python .mltoolkit/session.py \
  --data data/patients.csv --target readmitted_30d \
  --output-dir .mltoolkit --stage all \
  --sensitive-features race,sex,zip_code \
  --group-col race \
  --calibrate sigmoid --bootstrap 1000 \
  --decision-curve --optimize-threshold youden \
  --finalize

Regression with time-based CV and robust estimators