Stats
Actions
Tags
From data-annotation
Annotate a small dataset directly inside the Claude Code session — Claude reads each record, applies the locked schema, and writes structured annotations to disk. For datasets where standing up Label Studio or a Gemini batch job would be overkill (typically tens to low hundreds of records). Use when the user says "annotate this dataset", "label these records with Claude", "do the annotations yourself", or "small-scale annotation".
How this skill is triggered — by the user, by Claude, or both
Slash command
/data-annotation:annotate-with-claudeThe summary Claude sees in its skill listing — used to decide when to auto-load this skill
For small annotation jobs where the right tool is just Claude, a schema, and a JSONL file. No batch infra, no UI, no API key — Claude reads each record in turn, applies the schema, writes the output, and moves on. The user supervises (skim, spot-check, course-correct mid-run).
For small annotation jobs where the right tool is just Claude, a schema, and a JSONL file. No batch infra, no UI, no API key — Claude reads each record in turn, applies the schema, writes the output, and moves on. The user supervises (skim, spot-check, course-correct mid-run).
This skill complements scaffold-annotation-env (which sets up Gemini batch inference for larger runs). Choose based on scale and feedback loop:
| Situation | Use |
|---|---|
| ≤ ~200 records, schema may still need refinement, want tight feedback | annotate-with-claude (this skill) |
| Hundreds to thousands of records, schema is locked | scaffold-annotation-env |
| Mixed: a small calibration set first, then scale | This skill for the calibration set, then scaffold |
shape-dataset) to reshape first. Each record must have a stable id.schema.json produced by the schema-designer subagent. If absent, run schema-designer first; do not invent a schema mid-run.schema.json.<workspace>/annotation/claude-annotations.jsonl.schema.json and the guidelines into context.ids so the run is resumable.Before any records get annotated, draft an annotation workflow tailored to this dataset and write it as a SKILL.md inside the user's annotation repo. This makes the procedure inspectable, version-controlled, and reproducible — and means a future session (or a different annotator) can replay the exact process.
Where it lives: <repo>/.claude/skills/annotate-<dataset-slug>/SKILL.md if the user is in a Claude-aware repo, or <repo>/skills/annotate-<dataset-slug>/SKILL.md if they prefer a plain skills/ directory. Ask which they want; default to .claude/skills/.
What goes in it: the skill is the per-job operating manual, not a copy of this generic plugin skill. It must include:
name: annotate-<dataset-slug>, a one-line description that names the dataset and task.schema.json plus the inline label set / field shape so a reader doesn't need to context-switch.Show the proposed skill to the user before writing. Treat it as the contract for the run — if the user wants the procedure to change, edit the skill, don't just deviate silently. After the user approves, write it to disk and git add it (don't commit unless they ask — many users like to commit the skill alongside the first batch of annotations).
The rest of this workflow (calibration, bulk pass, output format, validation) executes against that committed skill. If the skill says "abstain when language is not English," that rule is binding for the run.
For the first 5–10 records, annotate slowly and visibly:
Calibration surfaces schema gaps fast — it's far cheaper to fix the schema after 5 mislabels than after 200. If the user corrects an annotation, also ask whether the schema or guidelines should be updated; if yes, update them and restart calibration on the corrected set.
After calibration is approved, switch to bulk mode:
Each line of the output JSONL:
{"id": "<record id>", "annotations": { ... matches schema ... }, "confidence": "high|medium|low", "rationale": "<one short line, optional>", "abstained": false, "annotator": "claude-<model>", "annotated_at": "<iso8601>"}
If abstained: true, the annotations field may be partial or null and the record is queued for human review.
When the run finishes (or the user stops):
schema.json using the validate_outputs.py helper from scaffold-annotation-env (or inline jsonschema).review-annotations for a sample audit, or move to hf-setup to publish.id is already in the output file unless the user explicitly asks for re-annotation.abstained: true and explain why. Inflated confidence is worse than honest gaps.npx claudepluginhub danielrosehill/claude-code-plugins --plugin data-annotationSearches MemPalace before answering questions about past work, people, projects, or prior decisions. Returns verbatim stored content instead of guessing from model memory.
Guides Payload CMS config (payload.config.ts), collections, fields, hooks, access control, APIs. Debugs validation errors, security, relationships, queries, transactions, hook behavior.
Implements vector databases with Pinecone, Weaviate, Qdrant, Milvus, pgvector for semantic search, RAG, recommendations, and similarity systems. Optimizes embeddings, indexing, and hybrid search.