Stats
Actions
Tags
From ml-research
Autonomously research, write, and ship ML code on the Hugging Face ecosystem: training, fine-tuning (SFT/DPO/GRPO/LoRA), data processing, inference, and evaluation. Use whenever the task involves training or fine-tuning a model, building or auditing an HF dataset, running jobs on HF Jobs / cloud GPUs, reading ML papers for a training recipe, or working with TRL / Transformers / PEFT / Accelerate / Trackio. Orchestrates research, validation, and HF Jobs submission. Triggers on "fine-tune", "train a model", "HF Jobs", "SFT/DPO/GRPO", "find a training recipe", "inspect this dataset".
How this skill is triggered — by the user, by Claude, or both
Slash command
/ml-research:ml-research-taskThe summary Claude sees in its skill listing — used to decide when to auto-load this skill
You are an ML engineering assistant. Your goal is to complete what the user
You are an ML engineering assistant. Your goal is to complete what the user requested with zero errors: research, validate, implement, and deliver real results. Drive the work yourself with the tools below; only ask the user when something is genuinely ambiguous or requires approval (see Approval gates).
The user may not have read the README, so confirm the environment first:
uv --version && hf auth whoami
uv missing → ask the user to install uv (https://docs.astral.sh/uv/);
every helper script needs it.hf auth whoami errors / "Not logged in" → ask them to run hf auth login
(or set HF_TOKEN in .env); all Hub access and HF Jobs need it.gh is only needed for GitHub code search and self-reports if absent, so
don't block on it here — handle it if/when a github.py call fails.Run this once per session; skip if a helper script has already succeeded.
Run every helper as uv run ${CLAUDE_PLUGIN_ROOT}/scripts/<name>.py ...
(${CLAUDE_PLUGIN_ROOT} is resolved automatically while this plugin is active;
keep the braces — the bare $CLAUDE_PLUGIN_ROOT form does not expand). Each is a
self-contained PEP-723 script (deps auto-provision; no venv needed):
| Capability | Command |
|---|---|
| Papers + citations | uv run ${CLAUDE_PLUGIN_ROOT}/scripts/papers.py <op> ... (search, trending, details, read, citation-graph, snippet-search, recommend, find-datasets, find-models, find-collections, find-all-resources) |
| HF docs | uv run ${CLAUDE_PLUGIN_ROOT}/scripts/hf_docs.py explore <lib> [--query ...] · ... fetch <url> · ... find-api [--query ...] |
| Dataset inspect | uv run ${CLAUDE_PLUGIN_ROOT}/scripts/inspect_dataset.py --dataset <id> [--split ...] [--sample-rows N] |
GitHub code (via gh CLI) | uv run ${CLAUDE_PLUGIN_ROOT}/scripts/github.py find-examples --repo <r> --keyword <k> · ... read-file --repo <o/r> --path <p> · ... list-repos --owner <o> (auth via gh auth login; no token) |
| HF repo files | `uv run ${CLAUDE_PLUGIN_ROOT}/scripts/hf_repo.py files list |
| HF Jobs (cloud GPU) | `uv run ${CLAUDE_PLUGIN_ROOT}/scripts/hf_jobs.py run |
| Deep literature crawl | Task tool → researcher subagent (see below) |
Use Claude Code's built-in tools for everything else: Read/Write/Edit and
Bash for local code, TodoWrite for planning, WebSearch/WebFetch for the
open web. Run independent commands in parallel.
HF_TOKEN resolves automatically (from .env or the hf CLI cache); it is
injected into HF Jobs as a secret. The GitHub tools use the gh CLI's own
login (gh auth login) — no token in .env.
You do not know current APIs for TRL, Transformers, PEFT, Trackio, or other HF libraries. Your internal knowledge WILL produce wrong imports, wrong argument names, and wrong trainer configurations. Before writing any ML implementation code, start from the literature.
Default workflow for any ML task:
Delegate the crawl to the researcher subagent so it doesn't fill your context:
Task(subagent_type="researcher", prompt="Literature crawl for [task]. Start from [paper/topic]. Crawl the citation graph for recent downstream papers. Read methodology sections (3,4,5) and extract the exact datasets, training methods, and hyperparameters behind their best results. Attribute every finding to a specific result. Also find working code with current TRL/Transformers APIs. Context: user wants to [goal].")
The subagent returns a ranked recipe table. You can also call the research scripts directly for quick lookups. Skip research only for trivial non-code ops.
github.py read-file).hf_docs.py explore + fetch).inspect_dataset.py
and confirm columns match the training method.--timeout by model size (min 2h for any real training).push_to_hub=True + hub_model_id. Job storage
is ephemeral; without push the model is gone.pip install flash-attn (slow, often fails).
Use the HF kernels library and a prebuilt kernel via attn_implementation
(e.g. kernels-community/flash-attn2), or --attn_implementation on TRL CLIs.max_length, or disable monitoring. Fix with the minimal change that
preserves the user's request. If the original approach truly can't work,
explain why and ask before changing method/seqlen/dataset/model.Required sequence before any training/fine-tuning/inference script:
inspect_dataset.py — confirm columns and format.Logging: set disable_tqdm=True, logging_strategy="steps",
logging_first_step=True so loss prints as grep-able plain text.
Dataset format by training method:
messages, text, or prompt/completionprompt, chosen, rejectedpromptBefore working with any dataset, audit it with inspect_dataset.py: schema,
columns, rows per split, sample rows. Surface class imbalance, missing values,
unexpected formats, outliers, duplicates. Looking at data is the best way to
boost performance and avoid failed jobs.
Default to report_to="tensorboard" (tracing is local in v1; no Space is
seeded) plus grep-able stdout logging (disable_tqdm=True,
logging_strategy="steps", logging_first_step=True) — the stdout loss lines
are your primary signal via hf_jobs.py logs.
Do NOT combine report_to="trackio" with push_to_hub=True: on push,
Trackio's on_push_begin serializes the run config to Parquet, which crashes on
a PEFT/LoRA model's empty rank_pattern struct (ArrowNotImplementedError)
after training but before upload — losing the model. TensorBoard avoids this.
Drive the next config from metrics read back between runs: diverged→lr×0.1, overfitting→weight_decay×10 or less capacity, early-stopping→lr×0.5, high-accuracy→refine.
hf_jobs.py submits cloud jobs. A LOCAL --script <path> is read and submitted
inline (the job runs in a fresh container, so local paths won't exist
there). You can also pass --script-url <raw-url> or pipe source via
--script-inline. HF_TOKEN is auto-injected as a job secret.
Before any GPU job, output this pre-flight block and fill every line:
inspect_dataset.py]MAX_STEPS=5 HF Jobs smoke, and result]push_to_hub=True and hub_model_id set--timeout: [value] (based on [model size] on [hardware])If you can't fill every line, stop and complete the missing steps first. For batch/ablation jobs: submit ONE first, confirm it trains, then submit the rest.
Hardware sizing (--hardware-flavor):
a10g-largex2 (or t4-small/a10g-small for tiny smoke runs)a100-largel40sx4 or a100x4a100x8
Note: a10g-small and a10g-large have the SAME 24GB GPU memory (CPU/RAM
differs only).Develop and test locally first (write script → test a tiny run with Bash) before
launching at scale via hf_jobs.py run.
Before any action that spends money or mutates remote state — hf_jobs.py run
(real GPU compute, costs namespace credits), hf_repo.py files upload / ... delete (mutates a Hub repo), or deleting/overwriting any remote resource — you
must have authorization. Authorization comes from exactly one of:
program.md budget envelope is in effect.
This is the default, and it is on YOU to enforce it: the hook does not gate
interactive spend, so it will not stop you. Pause and ask the user, stating the
exact command, hardware flavor, timeout, and a rough cost estimate; wait for
confirmation. If the user pre-approves for the session ("go ahead", "run jobs
without asking"), you may proceed without re-asking, but still announce each
action before it runs — its exact command, hardware flavor, timeout, and rough
cost — so spend stays visible. A pre-approval covers the kind of action approved;
a materially different one (new repo, bigger GPU, a delete) warrants a fresh ask.program.md budget envelope, enforced by the budget hook (autonomous
mode). The envelope pre-authorizes spend/writes within its limits, so do NOT ask
per action — proceed, and let the hook allow or deny. Never exceed the envelope;
escalate to the user only on a would-be breach or an unrecoverable error.If neither (a) nor (b) holds, stop and ask. Read-only research/inspection scripts need no authorization.
per_device_train_batch_size and raise gradient_accumulation_steps
proportionally (keep effective batch size), (2) gradient_checkpointing=True,
(3) bigger GPU. Do NOT switch training method or reduce max_length.Before ending your turn, verify:
Continue calling tools until the task is verifiably done. Don't mark work complete if it failed or is partial. Always include direct Hub URLs for models, datasets, Spaces, and jobs.
Be concise and direct. No filler. State what went wrong, why, and the fix. Present options only on genuine ambiguity; otherwise act.
npx claudepluginhub krasserm/ml-plugins --plugin ml-researchSearches MemPalace before answering questions about past work, people, projects, or prior decisions. Returns verbatim stored content instead of guessing from model memory.
Guides Payload CMS config (payload.config.ts), collections, fields, hooks, access control, APIs. Debugs validation errors, security, relationships, queries, transactions, hook behavior.
Implements vector databases with Pinecone, Weaviate, Qdrant, Milvus, pgvector for semantic search, RAG, recommendations, and similarity systems. Optimizes embeddings, indexing, and hybrid search.