Embeddings Search Skill
A Claude Code plugin that provides hybrid document search combining grep (literal matching) with semantic embeddings for comprehensive search across project documents.
Part of The-Focus-AI/claude-marketplace.
Features
- Hybrid Search: Combines literal text matching (grep) with semantic similarity (embeddings)
- Chunking Strategies: Sliding window, paragraph, or sentence-based chunking
- Metadata Extraction: Automatically extracts collections, document types, authors, and topics
- Taxonomy Discovery: Browse discovered collections, document types, authors, and topics
- Unified Index: Single
.embeddings/index.json file for fast loading
- Date & Metadata Filtering: Search within specific time ranges and filter by metadata
- Auto-indexing: Automatically indexes documents on first search
- Incremental Updates: Only re-indexes changed files
Installation
Install via Claude Code plugin marketplace or add directly:
claude --plugin-dir /path/to/embeddings-search-skill
Usage
The skill activates automatically when searching for documents by meaning:
- "Find documents about budget planning"
- "Search for discussions about the Q4 review"
- "Look for files related to authentication"
Manual CLI Usage
# Index a directory (default: sliding window strategy)
npx tsx src/cli.ts index .
# Index with specific strategy
npx tsx src/cli.ts index . --strategy paragraph
npx tsx src/cli.ts index . --strategy sentence
npx tsx src/cli.ts index . --strategy sliding --window 500 --overlap 150
# Search documents
npx tsx src/cli.ts search "budget concerns"
# Search with filters
npx tsx src/cli.ts search "love" --collection "jane austin"
npx tsx src/cli.ts search "meeting notes" --type report
npx tsx src/cli.ts search "quarterly review" --after 2024-01-01 --before 2024-04-01
# View discovered taxonomy
npx tsx src/cli.ts taxonomy .
# List files matching filters
npx tsx src/cli.ts files --collection "jane austin" --type novel
# View index statistics
npx tsx src/cli.ts stats .
Chunking Strategies
- sliding (default): Fixed-size windows with overlap. Best for precise retrieval.
- paragraph: Respects document structure (headers, paragraphs). Best for structured docs.
- sentence: Groups sentences up to target size. Best for natural language queries.
Metadata Extraction
The indexer automatically extracts metadata from:
- File paths: Collection names from directory structure
- Content patterns: Document type (novel, letter, report, essay, documentation)
- Headers: Title, author, date
- Text analysis: Topics/themes, mentioned people
Filtering Options
--collection <name> - Filter by collection (derived from directory names)--type <type> - Filter by document type (novel, letter, report, essay, documentation, notes)--author <name> - Filter by author--topic <topic> - Filter by topic (love, marriage, society, money, etc.)--path <pattern> - Filter by file path pattern--after <date> - Only documents modified after date--before <date> - Only documents modified before date
Supported File Types
Directly indexed:
.txt - Plain text.md - Markdown
Requires text extraction first:
.pdf - PDF documents.doc, .docx - Word documents.ppt, .pptx - PowerPoint presentations
For binary files, create a .txt sidecar with extracted text (e.g., report.pdf.txt).
Data Storage
Index is stored in .embeddings/index.json containing:
- File metadata (path, size, mtime, content hash)
- Document chunks with embeddings
- Extracted metadata (title, author, collection, topics, people)
- Taxonomy aggregations
Technical Details
- Embedding Model: Xenova/all-MiniLM-L6-v2 (~23MB, downloads on first use)
- Vector Dimensions: 384
- Similarity Metric: Cosine similarity
- Index Version: 5
- Storage: Single JSON index file
Dependencies
@xenova/transformers - Local embedding modelfast-glob - File discoverytsx - TypeScript execution
License
MIT
The-Focus-AI
