yapcode

yapcode

Talk to your laptop. Watch Claude Code write the code.

yapcode is one voice agent for all your Claude Code sessions. It becomes Claude's mouth and ears: you speak, and it drives real Claude Code sessions on your machine via tmux — starting them across your projects, sending instructions, approving permission prompts, running slash commands, and narrating the results back. A live terminal streams the actual Claude TUI to your browser and your phone, so you can watch (and take over by keyboard) at any time.

What it does

• 🎙️ Hands-free Claude Code — speak; the agent drives a real claude session and reads the answer back.
• 🖥️ Live terminal, anywhere — the Claude TUI streams to browser and phone via xterm.js; watch it work or grab the keyboard.
• 🔀 OpenAI Realtime or Gemini Live — pick your voice model (OpenAI direct or via Azure; Gemini has a free tier). Keys stay server-side.
• 📱 Code from mobile (on your local network, for now) — talk to Claude and watch the live terminal from anywhere in the house. Setup: use it from your phone.
• ✅ Approve by voice — say "yes", "allow", or "switch to auto mode" without touching the keyboard.
• 🤝 Co-drive — voice and keyboard share one tmux session; type and talk on the same session simultaneously.
• 🔁 Hand off in either direction — /voice-handoff continues a terminal session by voice; the printed tmux attach takes the keyboard on a voice one. Details: co-driving.
• 🖥️ Control the real TUI via voice — "run /init" · "interrupt that" · "what's on the screen?"
• 🗂️ Juggle projects — list, start, rename, and close sessions across your allowed folders, by name, entirely by voice.

[!WARNING] The backend executes real commands on your computer — it drives claude, which runs shell commands, edits files, and approves tool calls on your behalf. Read Security model (read this first) before you expose it beyond localhost.

---

Quickstart

Prerequisites

yapcode up checks these and tells you what's missing (full list):

• Claude Code, installed and logged in
• Voice API key — Gemini (free tier), OpenAI native, or OpenAI via Azure
• tmux
• Python 3.12+
• Node 20+

Installation

brew tap nithiink/yapcode && brew install yapcode   # macOS / Linuxbrew
yapcode up    # first run: setup wizard → opens http://localhost:3000

Or from source:

git clone https://github.com/nithiink/yapcode.git
cd yapcode
./bin/yapcode up    # same wizard; installs deps on first run

The first-run wizard asks for a voice key and the folder(s) the agent may edit; after that, yapcode up just launches.

---

Contents

• How it works
• Security model (read this first)
• What's in the box
• Prerequisites
• Install with Homebrew (recommended)
• Install from source
• Running
• Use it from your phone (local network)
• Windows (WSL2)
• Configuration reference
• Security & access control (details)
• Co-driving: voice + keyboard on one session
• Troubleshooting
• Architecture (for contributors)
• Contributing
• License

---

How it works

  you ──speak──▶  Voice agent          (OpenAI / Azure, or Gemini Live)
                    │  tool calls
                    ▼
                 Backend  (FastAPI, :8000)
                    │  tmux send-keys / capture-pane
                    ▼
                 Claude Code CLI  (real `claude` session in a tmux pane)
                    │
                    ▼
  browser/phone ◀── live xterm.js terminal  (Next.js frontend, :3000)