pidgin

pidgin

Grammarly in reverse. It makes your writing worse for humans and perfect for machines.

pidgin is a token-compression layer between you and your AI. You type like a normal person; the model receives a tightened version; nothing changes for you.

you type:    Hey, could you please send Sam this exact message: "the build is
             fixed, ship it" and please cc Alex on the git repository thread. Thanks!

model sees:  send Sam this exact message: "the build is fixed, ship it"
             and cc Alex on repo thread.

Notice what survived untouched: the quoted payload (byte-identical), every name, the meaning. That's the design constraint everything else serves.

Two directions, one codebook

Egress (default on): you type normal, pidgin compresses outbound. Deterministic only, provably meaning-preserving:

• Span freezing: quoted text, code fences, URLs, file paths, and emails are untouchable
• Filler strip: politeness and throat-clearing only ("could you please", greetings, trailing thanks); never modals or auxiliaries, so a musing can't become an order
• Transparent-code substitution: only codebook entries any frontier model reads without a glossary
• Closed-grammar templates: whole-message grammars compress by parsing, not paraphrase ("Could you please remind me in 8 days to call the bank?" -> remind in 8d: call the bank); extend yours in templates.yaml
• Re-quote dedup: paragraphs repeated byte-identical from earlier in the conversation become a short marker (provably lossless)
• Verification: every number and proper noun must survive into the compressed form, or your original text ships unchanged. Fallback-on-any-doubt is the rule.

Ingress: if you drift into shorthand yourself (most people do, fast), pidgin makes it safe. A glossary of your personal codes is injected so any model recovers your intent, and the action gate interrupts only when an action rides on an ambiguous reading: a misread question costs one clarifying turn, so questions never interrupt. Measured interrupt rate on real traffic: 0%.

The codebook is a plain YAML file of your shorthand, mined from your own chat history by miner.py (you approve every entry). Human-readable, diffable, shareable, portable to a different AI entirely. No weights, no lock-in.

Honest expectations

We had this design adversarially reviewed (two independent AI reviewers, full code access) before building, and the numbers below reflect that loop, not marketing:

• Egress on natural chatty messages: 10-25% of message tokens. Deterministic, zero meaning risk, zero latency. On already-terse messages it does nothing, by design.
• Ingress with adopted shorthand: up to ~60% (2.6x), measured with tiktoken on real messages. That ceiling needs you to type dense; pidgin makes that safe rather than forcing it.
• Your message is the smallest part of the API call. System prompts, history, and injected context dominate. pidgin's biggest single win in our own stack was compressing a 109-token scheduler boilerplate that rode every automated job, down to 41. Hunt your templates; cli.py audit shows the assembled-call picture so you measure the right denominator.
• What we deliberately did NOT build: an LLM that paraphrases your words in the hot path. A token-survival check is structurally blind to role reversal ("Alex pays Sam" vs "Sam pays Alex"), question-to-imperative flips, modality loss, and by-vs-to on numbers. Until that's solved, paraphrase compression stays out of the send path. Deterministic or nothing.
• Small local models misread dense text (measured: a 7B was intent-equivalent on 1 of 5 dense tasks), so pidgin's tier switch expands text for them instead. Compression is for models that can take it.

Install

Fastest path: paste this repo URL at your AI agent and say "install this". AGENTS.md has agent-directed instructions. Full walkthroughs per stack (including custom stacks like Codex or OpenClaw) live in docs/INSTALL.md.

Claude Code (plugin, ships its own hook and /pidgin command):

/plugin marketplace add nurbanasaurus/pidgin
/plugin install pidgin

Hermes gateway (hook + /pidgin command on Telegram/Discord/CLI):

git clone https://github.com/nurbanasaurus/pidgin ~/pidgin
~/pidgin/install.sh && hermes plugins enable pidgin-gate   # then restart the gateway

Any other stack: core.py is stdlib + PyYAML, three functions, no network. If your code can modify an outbound message or inject context, you can host pidgin; the wiring patterns (and the safety rules that come with them) are in docs/INSTALL.md.

Use