llmemory

llmemory

A high-performance document memory system with vector search capabilities for Python applications.

Overview

llmemory provides document search with:

• Complete Document Management API – List, retrieve, search, and manage documents without direct database access
• State-of-the-art retrieval using PostgreSQL with pgvector, hybrid BM25, multi-query expansion, and reranking
• Multi-language support with automatic language detection and normalization
• Hierarchical chunking & summaries with document-type specific configurations and optional auto-summaries
• Monitoring with Prometheus metrics and search diagnostics
• Reliable async Postgres foundation via pgdbm (connection pooling, migrations, schema support)

What's New

• 🔁 Multi-query expansion – Generate semantic + keyword variants automatically and fuse results with reciprocal rank fusion
• 🎯 Configurable reranking – Plug in OpenAI or cross-encoder rerankers (or use built-in heuristics) for higher precision on the final hit list
• 🧭 Query routing – Automatic answerability detection routes queries to best retrieval strategy
• 🎨 Contextual retrieval – Anthropic-style chunk enrichment with document context for improved semantic matching
• ⚙️ HNSW presets – Choose fast, balanced, or accurate profiles to tune pgvector index parameters and query-time ef_search
• 📝 Chunk summaries – Capture short, metadata-aware synopses during ingestion and surface them with every search hit
• 📈 Richer diagnostics – Search history now records query variants, latency breakdowns, rerank status, and summary usage for easy tuning

Why llmemory?

Applications with document search capabilities require:

• Vector embeddings for semantic understanding
• Efficient chunking that preserves context
• Hybrid search combining vectors and full-text
• Multi-tenant isolation for SaaS applications
• Performance optimization for large document sets

llmemory provides a solution.

Key Features

• 🚀 Fast Search – HNSW indexes for sub-100 ms vector searches, with multi-query expansion and optional cross-encoder reranking for harder queries
• 🌍 Multi-language – Automatic detection and processing for 14+ languages
• 📊 Smart Chunking – Document-type aware chunking with contextual enrichment, optional inline summaries, and hierarchical parent context
• 🔍 Hybrid Search – Combines vector and text search with reciprocal rank fusion, query routing, and rerank scores
• 📈 Observable – Built-in Prometheus metrics and detailed search diagnostics
• 🏢 Multi-tenant – Owner-based isolation for SaaS applications
• 🔌 Flexible Embeddings – Support for OpenAI and local embedding models

Quick Start

from llmemory import LLMemory, DocumentType, SearchType

# Initialize
memory = LLMemory(
    connection_string="postgresql://localhost/mydb",
    openai_api_key="sk-..."
)
await memory.initialize()

# Add a document - returns detailed results
result = await memory.add_document(
    owner_id="workspace-123",
    id_at_origin="user-456",
    document_name="project-report.pdf",
    document_type=DocumentType.REPORT,
    content="Your document content here..."
)
print(f"Created {result.chunks_created} chunks in {result.processing_time_ms}ms")

# List documents with filtering
docs = await memory.list_documents(
    owner_id="workspace-123",
    document_type=DocumentType.REPORT,
    metadata_filter={"status": "active"}
)

# Search with document metadata
results = await memory.search_with_documents(
    owner_id="workspace-123",
    query_text="project timeline",
    search_type=SearchType.HYBRID
)
for result in results.results:
    print(f"Found in: {result.document_name} - {result.content[:100]}...")

# Get statistics
stats = await memory.get_statistics("workspace-123")
print(f"Total: {stats.document_count} docs, {stats.chunk_count} chunks")

Installation

# Install using uv (recommended)
uv add llmemory

# Or using pip
pip install llmemory

Or with optional dependencies:

# Using uv
uv add "llmemory[monitoring]"  # For Prometheus metrics
uv add "llmemory[cache]"       # For Redis caching
uv add "llmemory[local]"       # For local embeddings
uv add "llmemory[reranker-local]"  # For local cross-encoder reranking
uv add "llmemory[bench]"       # For BEIR benchmarking harness