datoon

datoon

datoon is a pragmatic smart TOON gateway for LLM data workloads: it converts JSON payloads to TOON only when conversion is likely to improve token efficiency.

Raw JSON is often verbose in prompts. TOON can save tokens, but blind conversion can also make payloads worse. datoon adds a decision layer so pipelines can convert when savings are meaningful, skip when structure is a poor TOON fit, and always report exactly why the decision was made.

Before / After

Before: JSON payload in the prompt

{"users":[{"id":1,"name":"Ada","role":"admin"},{"id":2,"name":"Lin","role":"analyst"},{"id":3,"name":"Grace","role":"viewer"}]}

After: datoon converts only because policy says it should

users[3]{id,name,role}:
  1,Ada,admin
  2,Lin,analyst
  3,Grace,viewer

{
  "decision": "convert",
  "reason": "Estimated savings 44.19% (threshold 15.00%).",
  "input_token_estimate": 43,
  "output_token_estimate": 24
}

Same records, same keys, smaller prompt payload. If the structure is non-uniform, tiny, deeply nested, or conversion saves too little, datoon keeps JSON instead.

Auto mode returns compact JSON instead of TOON when the payload is not a good fit: no uniform object arrays, fewer than the configured minimum rows, nesting deeper than --max-depth, estimated savings below --min-savings, or a missing/unavailable TOON CLI dependency. Invalid JSON still fails loudly, and --force bypasses gating for experiments but does not hide conversion failures.

---

Contents

• Setup
• Install
• Quick Start
• Python API
• MCP Server
• Claude Code Plugin
• CLI Reference
• Benchmarks
• Development
• Docs

---

⚡ Setup

One-command setup — installs all dependencies and registers datoon globally so it's available in any terminal:

./setup.sh

What it does:

1. Checks Python 3.12+ (fails fast if missing)
2. Installs uv if not present
3. Warns if Node.js is missing (required for TOON conversion)
4. Runs uv sync --extra dev
5. Registers the datoon CLI globally via uv tool install --editable .
6. Adds ~/.local/bin to your shell profile if needed

After setup:

datoon --help
echo '{"users":[{"id":1,"name":"Ada"}]}' | datoon --report-stdout

Full install options for Python, CLI, MCP, Claude Code, and Codex live in INSTALL.md. For integration installs, preview first with ./install.sh --dry-run, then use ./install.sh --install --target <claude|codex|mcp>. The installer only changes selected local integration config: Claude Code plugin state through the claude CLI, Codex marketplace JSON, and/or an MCP JSON config. See INSTALL.md for exact paths, backups, privacy notes, and troubleshooting.

---

📦 Install

# via uv (recommended)
uv add datoon

# via pip
pip install datoon

# with optional tiktoken-based token counting
pip install "datoon[tokens]"

# with MCP server support
pip install "datoon[mcp]"

Requires Python 3.12+. TOON conversion requires Node.js with npx in PATH.

datoon itself processes payloads locally. The Python package has no required runtime dependencies; optional extras add token estimation (tiktoken) or MCP server support (mcp). The converter invokes npx --yes @toon-format/cli@2 only when auto policy reaches the conversion step or when --force is used.

---

🚀 Quick Start

stdin:

echo '{"users":[{"id":1,"name":"Ada"},{"id":2,"name":"Lin"}]}' | datoon --report-stdout

file:

datoon ./input.json -o ./output.toon --report ./report.json

force conversion: