claude-lcm

Claude LCM — Lossless Context Management

A Claude Code skill that ports the LCM architecture (from the Voltropy paper) natively into Claude Code using SQLite + Python scripts. No OpenClaw required.

Core Concepts

LCM maintains two states:

• Immutable Store — Every message, tool result, and assistant response, persisted verbatim in SQLite. Never modified. Source of truth.
• Active Context — What's actually sent to the model each turn: recent raw messages
  • precomputed summary nodes from the DAG.

As sessions grow, older messages are compacted into Summary Nodes in a hierarchical DAG. Originals are always retained. Any summary can be expanded back to source material.

When You Are Invoked

You are operating as the LCM skill. Your responsibilities:

1. Initialize the LCM store if it doesn't exist for this session
2. Decide whether compaction is needed (check token thresholds)
3. Compact older messages into summary nodes using the three-level escalation protocol
4. Answer recall queries using lcm_grep and lcm_expand_query
5. Process large datasets using lcm_llm_map or lcm_agentic_map
6. Handoff cleanly between sessions via checkpoint files

Read references/architecture.md for the full DAG and compaction design. Read references/tools.md for tool interface specifications. Read references/prompts.md for depth-aware summarization prompts.

How to Use LCM

Automatic (no action needed):

• Session init/resume, message ingestion, tool capture, compaction, checkpoints — all handled by hooks.
• You don't need to call lcm_init, lcm_ingest, or lcm_checkpoint manually.

On-demand (you invoke these):

• Search history → lcm_grep.py
• Recall a decision → lcm_expand_query.py
• Inspect a summary → lcm_describe.py
• Check context health → lcm_status.py
• Process a dataset → lcm_llm_map.py / lcm_agentic_map.py

Script Location

The $LCM_SCRIPTS environment variable is set automatically by the SessionStart hook. Use it in all Bash tool calls:

python3 "$LCM_SCRIPTS/lcm_grep.py" "your query"

If $LCM_SCRIPTS is not set (e.g. running outside the plugin), use the plugin root:

LCM_SCRIPTS="$CLAUDE_PLUGIN_ROOT/scripts"

Quick Start

Initialize for a new session

python3 "$LCM_SCRIPTS/lcm_init.py"

This creates ~/.claude-lcm/lcm.db with the full schema and prints the session ID.

Check context health

python3 "$LCM_SCRIPTS/lcm_status.py"

Prints: message count, estimated token usage, compaction threshold status, and whether a compaction pass is recommended.

Compact context

python3 "$LCM_SCRIPTS/lcm_compact.py"

Runs a leaf compaction pass over messages outside the fresh tail. If summaries are accumulating, also runs a condensation pass.

Search history

python3 "$LCM_SCRIPTS/lcm_grep.py" "database migration"
python3 "$LCM_SCRIPTS/lcm_grep.py" "config threshold" --scope summaries

Expand a summary

python3 "$LCM_SCRIPTS/lcm_expand.py" sum_abc123

Returns the source messages behind a summary node. Sub-agent only — main agent should use lcm_expand_query.py instead.

Process a dataset (LLM-Map)

python3 "$LCM_SCRIPTS/lcm_llm_map.py" \
  --input data.jsonl \
  --output results.jsonl \
  --prompt "Extract entities from this text" \
  --schema schema.json \
  --concurrency 16

Process with multi-step reasoning (Agentic-Map)

python3 "$LCM_SCRIPTS/lcm_agentic_map.py" \
  --input tasks.jsonl \
  --output results.jsonl \
  --prompt "Analyze this codebase" \
  --concurrency 4

Compaction Protocol

Always follow the Three-Level Escalation when compacting:

Level	Strategy	Fallback condition
1	LLM summarize, preserve details	Output longer than input
2	LLM summarize, bullet points only, half target tokens	Still longer than input
3	Deterministic truncation to 512 tokens	Never fails

Escalate automatically. Never leave a compaction pass in a failed state.

Fresh Tail Protection

The last 32 messages (configurable via LCM_FRESH_TAIL_COUNT) are never compacted. This ensures the model always has enough recent context for continuity.

Token Thresholds

Variable	Default	Meaning
LCM_CONTEXT_THRESHOLD	0.75	Fraction of window that triggers compaction
LCM_FRESH_TAIL_COUNT	32	Messages protected from compaction
LCM_LEAF_CHUNK_TOKENS	20000	Max source tokens per leaf chunk
LCM_LEAF_TARGET_TOKENS	1200	Target tokens for leaf summaries
LCM_CONDENSED_TARGET_TOKENS	2000	Target tokens for condensed summaries
LCM_LARGE_FILE_THRESHOLD	25000	Files above this stored externally
LCM_SUMMARY_MODEL	(inherit parent)	Model for summarization (empty = parent model)

DAG Structure

Raw messages → Leaf summaries (depth 0) → Condensed (depth 1) → Condensed (depth 2) → ...

Each level uses a different summarization prompt strategy (see references/prompts.md):

• Depth 0: Narrative with timestamps, file ops, decisions preserved
• Depth 1: Chronological session summary, deduplicated
• Depth 2: Arc-focused — goals, outcomes, what carries forward
• Depth 3+: Durable context only — key decisions, relationships, lessons

Session Handoff

Before ending a long session, always write a checkpoint:

python3 "$LCM_SCRIPTS/lcm_checkpoint.py"

This writes .lcm-checkpoint.md to the workspace root. On resume:

python3 "$LCM_SCRIPTS/lcm_resume.py"

Reads the checkpoint and reconstructs working state from the immutable store.

Large File Handling

Files over LCM_LARGE_FILE_THRESHOLD tokens are stored externally and replaced in context with a compact Exploration Summary. The agent retains awareness of the file without loading it. Use lcm_describe.py to inspect stored files.

Sub-agent Scope Guard

When spawning sub-agents for expansion tasks, each must declare:

• delegated_scope: what slice of work it's handling
• kept_work: what the caller retains

If a sub-agent cannot articulate kept_work, it must do the work directly. This prevents infinite delegation chains.