The best local backend for Claude Code.
Claude Code runs against any HTTP endpoint that speaks the Anthropic Messages API. Four serious options: ToolPiper, Ollama, vLLM, LM Studio. They are not equivalent. Here is what is actually different.
The short version
- Pick Ollama / vLLM / LM Studio if you want a single local model server and nothing else. They speak the protocol; that is the feature.
- Pick ToolPiper if you want Claude Code to move across local + cloud providers in the same conversation, with over 450 MCP tools, voice, browser automation, and Keychain-locked BYOK, from two switches: Tools and Engine.
- ToolPiper is the only backend that runs in your normal
claudewith one switch, leaves open sessions where they are, and states each model's own compaction window. A local model needs a trained window of at least 98,304 tokens and enough memory, and a compaction on a large one takes minutes.
Feature matrix
| Feature | ToolPiper | Ollama | vLLM | LM Studio |
|---|---|---|---|---|
| Zero-config Claude Code setup | One switch on Home: new sessions of your normal claude run on your model | Manual env vars | Manual env vars + DEFAULT_*_MODEL dance | Manual env vars + DEFAULT_*_MODEL dance |
| Conversational provider switching | "Run Claude Code on my local Qwen" works in chat (`claude_code_engine_set`) | No | No | No |
| Speak to Claude (STT + TTS + voice clone) | Yes | No | No | Dictation only (Bionic voice keyboard) |
| Anthropic Messages API (`POST /v1/messages`) | Native, with provider switching | No | Yes (since v0.7) | Yes (since 0.4.1) |
| OpenAI Chat Completions (`/v1/chat/completions`) | Yes | Yes | Yes | Yes |
| Streaming with cancellation | <200 ms upstream abort on Esc | Yes | Yes | Yes |
| Tool use round-trip | Anthropic + OpenAI shapes | OpenAI only | OpenAI only | Anthropic + OpenAI shapes |
| Inference backends | 9 (llama.cpp, Apple Intelligence, FluidAudio, MLX Audio, Apple Vision, …) | 1 (llama.cpp) | 1 (vLLM engine) | 2 (llama.cpp, MLX) |
| Apple Intelligence (Neural Engine) backend | Yes, first to ship it | No | No | No |
| BYOK cloud proxy (OpenAI / Anthropic / Gemini / etc.) | Yes (Keychain-locked, server-side injection) | No | No | No |
| Cross-provider routing in one base URL | Yes (local and cloud in one `/model` list) | No | No | No |
| Each model compacts at its own window | Yes, stated per `/model` row in Claude Code's settings | By hand | By hand | By hand |
| Open Claude Code sessions left where they are | Yes, only new sessions start on the engine | Env vars reach one shell | Env vars reach one shell | Env vars reach one shell |
| Native /model picker integration | Yes (writes `modelPicker` to Claude Code's user settings, and puts the file back on Off) | No | No | No |
| MCP server bundled | over 450 tools: browser, files, system, video, voice | No | No | No |
| MCP toolbelt injection (when client has no tools) | Header-gated, PiperMatch-retrieved | No | No | No |
| Browser automation (CDP) | 17 AX-native tools | No | No | No |
| OCR (Apple Vision) | Yes | No | No | No |
| Image / video upscale on Neural Engine | PiperSR (44 fps real-time 2× video) | No | No | No |
| Resource intelligence (auto model eviction) | Real-time RAM monitoring + eviction | Reads RAM once at startup | Manual | Not compared here |
| Distribution | Native macOS DMG | Homebrew + CLI | Python wheel + GPU container | Installers for macOS, Windows and Linux |
| Sandboxing | Native, signed | No | No | Not documented |
Performance baselines
Performance numbers depend heavily on your Mac, your model, and your prompt mix. ToolPiper ships scripts/anthropic-perf-baseline.ts: a harness that captures p50 / p95 TTFT and tokens/sec across every provider you have configured, including Ollama / vLLM / LM Studio if you wire them as endpoints. The output lives at docs/features/anthropic-perf-baseline.md and updates on every run. The numbers below are workload definitions; the live comparison is in the repo.
| Workload | ToolPiper | Definition |
|---|---|---|
| Single-turn 2K prompt (TTFT, p50) | See live numbers | Captured per `EndpointConfig` by `scripts/anthropic-perf-baseline.ts`. Runs across every provider you have configured. |
| Multi-turn 20K with tools (tokens/sec, p50) | See live numbers | Same harness (Phase 7 deliverable), populated on every fresh ToolPiper run. |
| Tool-heavy 50K (TTFT, p95) | See live numbers | Skipped automatically on backends whose context window is below 50K; annotated `context_overflow` in the output. |
Pick a recipe
Switch models with /model
Local and cloud in one /model list, or ask for it in chat. No env vars.
Use over 450 tools on a small local model
PiperMatch picks the right subset per turn and ships compact schemas so a tight-context local model can still reach the full catalog.
Local · 98K · Tool useLocal Qwen
Qwen 3.8 27B on llama.cpp via Metal GPU, with a 98K window for Claude Code.
BYOK · KeychainOpenAI with your key
gpt-4o through Claude Code, billed against your account, key locked in Keychain.
Try it
Download ToolPiper, choose a language model on Home, turn on Engine in the Claude Code row, and start a new claude.