Voice Chat With Ollama on Mac: Add STT and TTS to Any Local Model

Ollama runs language models. It doesn't listen and it doesn't speak. Type a question in the terminal, read the answer on screen. That's the entire interaction model.

Voice changes what local AI feels like. Instead of typing and reading, you talk and listen. The model becomes a conversational partner instead of a text box. But getting there requires three separate AI models working together, and Ollama only handles one of them. ToolPiper ships all three - and runs the chat model itself on its built-in llama.cpp engine, so Ollama is optional in the chain, not the foundation.

What does voice chat with a local model actually require?

Three models, running in sequence, every time you speak:

Speech-to-text (STT). Your voice goes in, a text transcription comes out. This needs a dedicated model - Whisper, Parakeet, or similar. Ollama doesn't include one.

Language model (LLM). The transcribed text goes to your chat model. Llama 3.2, Qwen 3.5, Mistral, DeepSeek - any GGUF model works here, running on ToolPiper's built-in engine or, if you already have one set up, an Ollama instance. This is the one piece Ollama covers.

Text-to-speech (TTS). The model's text response gets converted to audio. Another dedicated model - PocketTTS, Soprano, Orpheus, or similar. Ollama doesn't include this either.

The hard part isn't running each model. It's coordinating them. The STT output needs to feed into the LLM prompt. The LLM response needs to stream into the TTS engine as tokens arrive, not after the full response completes. Latency between stages compounds - if each handoff adds 500ms, the conversation feels broken.

You could wire this together manually with Python scripts, a Whisper server, and a TTS service. Some people do. It takes hours of setup, and the result is fragile.

How does ToolPiper add voice to Ollama models?

The same way it adds voice to any local model - the chain doesn't care where the LLM runs. ToolPiper ships STT, LLM, and TTS as built-in backends, all running on Apple Silicon hardware acceleration. The tp-local-voice-chat pipeline template wires all three together in a pre-configured workflow, with the chat model on ToolPiper's engine by default and an existing Ollama instance as the optional swap.

The speech-to-text backend uses Parakeet v3, running on Apple's Neural Engine. It transcribes in real-time on M-series chips, and it's free. The language model runs on ToolPiper's embedded upstream llama-server (build b9533) - or you can point the LLM block at an Ollama instance you already have running. The text-to-speech backend offers three engines, all part of Pro ($10/month):

PocketTTS - runs on the Neural Engine. Fastest option, near-instant generation. Default voice: Cosette (female). Good for conversational pace where you want the response to start immediately.

Soprano - runs on Metal GPU. Higher audio quality, slightly more latency. Default voice: Tara (female). Better for longer responses where you want the voice to sound more natural.

Orpheus - expressive model with emotional range. Default voice: Tara (female). Best for content creation and narration. Overkill for quick Q&A, worth it for anything where the voice quality matters.

All three TTS engines run entirely on your Mac. No audio leaves the device. To be precise about what costs money: speech-to-text and the chat model are free, no account required; the speaking half of the conversation - TTS - is what Pro ($10/month) adds.

How do you set up voice chat with Ollama on Mac?

Install ToolPiper - a starter model downloads on first launch, so the chat half works before you configure anything. Then open the tp-local-voice-chat template. The pipeline is three blocks connected in sequence: microphone input flows to STT, the transcript flows to the LLM, and the response flows to TTS. ModelPiper's pipeline builder shows this as a visual graph you can inspect and customize.

If you already have Ollama running with models downloaded, ToolPiper connects to it as an external provider. Your pulled models appear in the pipeline's LLM block alongside ToolPiper's native ones - a one-time setting, not a prerequisite. The full comparison covers whether keeping Ollama in the chain still makes sense.

Push-to-talk vs continuous listening

Two input modes. Push-to-talk activates the microphone when you hold a button (or a keyboard shortcut) and stops when you release. Continuous listening keeps the microphone open and uses silence detection to determine when you've finished speaking.

Push-to-talk is more predictable. You control exactly when the model hears you. Continuous listening is more natural for extended conversations but occasionally triggers on background noise. We default to push-to-talk for the pipeline template.

What does the latency actually look like?

Voice chat latency is the sum of three stages. We measured each on an M2 Max with 32GB, using Qwen 3.5 3B (Q4) for chat:

STT (Parakeet v3): A typical spoken sentence (5-10 words) transcribes in about 400ms. Parakeet runs on the Neural Engine, which is a separate processor from the GPU - so transcription doesn't compete with the LLM for Metal compute time.

LLM (3B model, Q4): Time to first token averages about 300ms in our testing. Tokens stream as they generate, and the TTS engine picks up partial output - it doesn't wait for the full response to complete.

TTS (PocketTTS): First audio plays about 350ms after receiving text input. Because of the streaming handoff, the user hears audio before the LLM finishes generating its full response.

Total round-trip: About 1.5 seconds from the end of your sentence to the first word of the spoken response with a 3B model on M2 Max. With a 7B model, the LLM's time-to-first-token roughly doubles, pushing total latency to about 2-2.5 seconds. A 13B model pushes it to 3-4 seconds.

What that feels like in practice: you stop talking, there's a beat of silence, and then the model starts speaking. With a 3B model, the pause is short enough that it feels like the model is formulating a thought. With a 13B model, the pause is noticeable - you start wondering if something broke before the first word arrives. For comparison, ChatGPT's voice mode typically responds in under a second, running on optimized server hardware. Local voice chat on consumer hardware can't match that speed, but it runs entirely on-device with no internet connection and no data leaving your Mac.

What are the limitations of local voice chat?

Latency is real. Cloud voice assistants like ChatGPT's voice mode use optimized infrastructure and voice-native models to achieve sub-second response times. Local models on consumer hardware can't match that speed, especially with larger models. The 1-2 second pause with a 3B model is the floor, not the ceiling.

Three models in memory simultaneously. STT, LLM, and TTS each need RAM. Parakeet v3 uses roughly 500MB. A 3B chat model at Q4 uses about 2GB. PocketTTS uses about 300MB. Total: roughly 3GB for the smallest viable voice chat setup. On an 8GB Mac, that leaves tight headroom. On 16GB or more, it's comfortable. For the full picture on running multiple models at once, see running multiple Ollama models on Mac.

No interruption handling. If the model is speaking and you start talking, the current implementation doesn't stop the TTS output mid-sentence. You need to wait for it to finish or manually stop playback. This is a known limitation we're working to improve.

Ambient noise sensitivity. Continuous listening mode can false-trigger on background audio - music, other people talking, keyboard sounds. Push-to-talk avoids this entirely, which is why it's the default.

For most conversational AI tasks - brainstorming, dictation review, Q&A while your hands are busy - local voice chat is good enough that you stop reaching for the keyboard. For rapid-fire dialogue where sub-second latency matters, cloud voice modes are still faster.

Download ToolPiper at modelpiper.com and try the tp-local-voice-chat pipeline template - the starter model covers the chat block, and your existing Ollama models plug in if you have them.

This is part of our Ollama series. See also: Voice Chat on Mac With Local AI for the general guide to local voice conversation. Next: Ollama Pipelines on Mac - chain models in a visual workflow.