ds

Extract and categorize learnings from completed experiments into docs/ds/learnings/ for future retrieval

Profile a dataset for structure, quality, distributions, and anomalies, then output an EDA report

Design an ML experiment with hypothesis, split strategy, leakage check, and evaluation plan

Frame a data science problem and plan the approach, surfacing relevant past learnings

Clean, validate, and transform raw data using automated preprocessing pipelines

Translate business questions into DS problems with target variables, metrics, and constraints. Use when starting a project or when the objective needs sharpening.

Define hypothesis, variables, split strategy, baselines, and comparison protocol. Use before running an experiment to lock down methodology.

Compute metrics, slice by subgroups, check calibration, and flag fairness gaps. Use after training to decide ship/iterate/abandon.

Evaluate whether an ML model is ready for production deployment by checking infrastructure, monitoring, rollback, and operational requirements. Use before shipping a model to production.

Extract reusable insights from experiment results and write them as searchable learning documents. Use at project end to capture what worked, failed, and surprised.

Aeon API patterns for time series machine learning -- classification, regression, clustering, anomaly detection, segmentation, and similarity search. Use when /ds:experiment needs time-series-specific ML algorithms (ROCKET, InceptionTime, DTW classifiers), or /ds:eda needs temporal feature extraction (Catch22, ROCKET features) or change point detection. For classical statistical forecasting (ARIMA/SARIMAX) use statsmodels; for tabular ML pipelines use scikit-learn; for visualization use matplotlib.

Pre-model data preparation pipelines for cleaning, validation, transformation, and ETL orchestration. Use when raw data needs deduplication, schema validation, format conversion, or quality assurance before EDA or modeling.

Data quality validation with Great Expectations, dbt tests, and data contracts. Use when building formal validation rules, expectation suites, or data contracts for repeatable quality gates.

Systematic exploratory data analysis checklist covering structure, quality, distributions, relationships, and target analysis. Use when starting EDA on any dataset.

Standard format for logging ML experiments including hypothesis, config, results, and learnings. Use when running experiments to maintain a consistent record.