Arcaneum
CLI tools and Claude Code plugins for semantic and full-text search.
pipx install "https://github.com/cwensel/arcaneum/releases/download/v0.8.0/arcaneum-0.8.0-py3-none-any.whl"
Overview
Arcaneum helps you discover and understand project dependencies, documentation, and reference implementations.
By indexing libraries, frameworks, and technical papers, you can semantically search for patterns, APIs,
and concepts when building new projects. Works especially well with the
RDR (Recommendation Data Record) model for AI-assisted development planning.
The system supports PDF documents and source code with git-aware, AST-based chunking.
Currently Available:
- Semantic search with Qdrant (vector embeddings)
- Full-text search with MeiliSearch (exact phrase matching)
- Dual indexing workflow for comprehensive search
Features
Search Capabilities
- Semantic Search (Qdrant): Find conceptually similar content using vector embeddings
- Full-Text Search (MeiliSearch): Exact phrase matching, keyword search, and typo-tolerant queries
Indexing
- PDF Indexing: OCR support for scanned documents, page-level metadata, parallel processing
- Source Code Indexing: Git-aware with AST chunking, multi-branch support, 165+ languages
- Markdown Indexing: YAML frontmatter extraction, semantic chunking, incremental sync
- Dual Indexing: Single command to index to both search engines
- Performance Tuning: Granular control over workers, batch sizes, and process priority via
arc corpus sync --max-embedding-batch, --text-workers, --cpu-workers, and
single-system indexing flags such as --embedding-batch-size and --process-priority
Multiple Embedding Models
- arctic-m (768D) - DEFAULT for PDFs/markdown - stable FastEmbed retrieval model
- stella (1024D) - High-quality opt-in document model, requires
arcaneum[sentence-transformers]
- mxbai-large (1024D) - High-quality FastEmbed document model
- jina-code (768D) - DEFAULT for code - stable FastEmbed code model
- jina-code-st (768D) - Legacy SentenceTransformers code path, requires
arcaneum[sentence-transformers]
- jina-code-0.5b (896D) - Higher-quality opt-in code model, 32K context, requires
arcaneum[sentence-transformers]
- jina-code-1.5b (1536D) - Highest quality code embeddings, SOTA Sept 2025, requires
arcaneum[sentence-transformers]
- codesage-large (1024D) - CodeSage V2, 9 programming languages, requires
arcaneum[sentence-transformers]
- bge-large (1024D) - BGE large embeddings, balanced performance
- jina-v3 (1024D) - Multilingual embeddings with extended 8K context
- bge-base (768D) - BGE base embeddings, balanced performance and speed
- bge-small (384D) - BGE small embeddings, fastest for size-constrained scenarios
See arc models list for complete model information and recommendations.
Use arc models list --json for the LLM-readable catalog: it includes backend,
recommended and default corpus uses, support/risk tier, prompt policy, context
limit, hardware support, runtime-aligned batch guidance, and reindex warnings.
Arcaneum records each collection's embedding prompt policy when it is indexed. Reindex a
corpus after changing model query/document prompts, tasks, or prompt-aware model defaults;
semantic search rejects collections whose stored prompt policy no longer matches the current
model registry.
GPU Acceleration
- CPU is the default for the most stable indexing behavior
- Supports Apple Silicon (MPS) and NVIDIA GPUs (CUDA)
- Use
--gpu to opt into accelerator embedding
- FastEmbed/CoreML on Apple Silicon is experimental and requires
ARC_EXPERIMENTAL_COREML=1
CLI-First Design
- All operations via command-line interface
- JSON output mode for automation
- Structured error messages with exit codes
- Python >= 3.12 required
Claude Code Integration
- Slash commands for all operations (
/arc:search, /arc:index, /arc:collection, etc.)
- Discoverable via
/help or /commands in Claude Code
- No MCP overhead - direct CLI execution
Quick Start
Get started with Arcaneum in just a few commands:
# 1. Install
pipx install "https://github.com/cwensel/arcaneum/releases/download/v0.8.0/arcaneum-0.8.0-py3-none-any.whl"
# 2. Install Claude Code plugin (optional, in Claude Code)
# /plugin install cwensel/arcaneum
# 3. Verify and start services
arc doctor
arc container start
# 4. Create a corpus and sync content (indexes to both Qdrant and MeiliSearch)
arc corpus create Frameworks --type code
arc corpus sync Frameworks ~/libs/fastapi ~/libs/sqlalchemy
