Voice AI on Mac: Transcription, Speech, and Conversation — All Local

The cloud voice model is broken

Every word you speak to a cloud transcription service is biometric data on someone else's servers. Your voice is as unique as a fingerprint. Pitch, cadence, timbre, the micro-hesitations between words, the way you emphasize syllables when you are being precise versus casual — all of it is a biometric signature that identifies you with the same certainty as a retinal scan. And every cloud voice service in the market today requires you to hand that signature over.

Otter.ai processes your meeting audio on their infrastructure. ElevenLabs stores the voice samples you upload for cloning. Google's Speech API logs requests by default. When these companies say they handle your data responsibly, they mean according to their current policies, which they can change with a terms-of-service update you will never read. The cloud model for voice AI is fundamentally at odds with the sensitivity of the data it processes. No other category of AI involves handing over something so personally identifying with so little friction.

This is not an abstract privacy concern. A leaked voice sample can be used for deepfake fraud. A compromised transcription database exposes the contents of every conversation that passed through it — legal discussions, medical consultations, HR conversations, board meetings. The risk is not that these companies intend harm. The risk is that biometric data, once collected, becomes a permanent liability. It cannot be rotated like a password. It cannot be revoked like an API key. Your voice is your voice for life.

The alternative is local processing. Every model in this article runs entirely on your Mac. Audio never leaves your machine. Transcripts exist only on your disk. Cloned voice models are stored in your memory and nowhere else. This is not a feature. It is a prerequisite for handling voice data responsibly.

With that context, here is what voice AI actually encompasses and why it matters that all of it can now run locally.

Speech-to-text (STT) converts spoken audio into written text. Modern STT models like Parakeet and Whisper are neural networks trained on hundreds of thousands of hours of transcribed speech. They handle accents, background noise, filler words, and domain-specific vocabulary with accuracy that was impossible five years ago. On Apple Silicon, STT runs on the Neural Engine, dedicated silicon designed specifically for ML inference. The Neural Engine operates independently from the GPU, meaning transcription does not compete with other AI workloads for resources.

Text-to-speech (TTS) synthesizes natural-sounding speech from written text. The generation of 2015, rule-based and robotic, has been replaced by neural TTS models that produce speech with natural pacing, emphasis, and intonation. On Mac, TTS models run on either the Neural Engine (FluidAudio's PocketTTS) or the Metal GPU (MLX Audio's Soprano, Orpheus, and Qwen3 TTS). The quality gap between local and cloud TTS has narrowed dramatically since 2024.

Voice cloning replicates a specific person's voice from a short audio sample. The model encodes pitch, cadence, timbre, and speaking patterns from the reference audio, then synthesizes new speech in that voice from any text input. As of March 2026, local voice cloning requires as little as 10-30 seconds of clear speech. The entire process runs on your GPU with no biometric data uploaded anywhere.

Voice assistants and conversation chain STT, a language model (LLM), and TTS together into a spoken dialogue loop. You speak, the AI transcribes, reasons, and responds with synthesized speech. Apple Silicon's architecture makes this uniquely efficient: STT runs on the Neural Engine, the LLM runs on the Metal GPU, and TTS runs on whichever hardware is available. Three models on three different processors, all running simultaneously without contention.

The state of the art (April 2026)

Voice AI on local hardware has crossed a threshold in the past year. The models are good enough, the hardware is fast enough, and the integration is mature enough that local processing rivals cloud services for most practical use cases. Here is what the landscape looks like right now.

Speech-to-text

Parakeet V3 is currently the fastest local STT model on Mac, running at approximately 210x realtime on the Neural Engine via FluidAudio. A 30-minute recording transcribes in under 10 seconds. The model supports 25 European languages with automatic detection and delivers Whisper-class accuracy for clear speech. NVIDIA released the Parakeet TDT architecture through their NeMo toolkit, and the CoreML-optimized variant that runs via FluidAudio represents the best speed-to-accuracy ratio available on Apple Silicon as of this writing.

OpenAI's Whisper remains the reference point for local STT quality. Whisper large-v3 (1.5B parameters) is the accuracy benchmark, but it runs at roughly 5-10x realtime on Mac GPUs via whisper.cpp, significantly slower than Parakeet on the Neural Engine. The whisper.cpp community has pushed optimization far, including Metal acceleration and CoreML conversion, but the fundamental architecture favors GPU execution over ANE execution.

Distil-Whisper variants (distil-large-v3) offer a middle ground: 756M parameters, roughly 6x faster than the full model, with accuracy within 1% on most benchmarks. These run well on Mac GPUs but still cannot match Parakeet's Neural Engine throughput.

Apple's built-in dictation engine ships on every Mac but offers lower accuracy than Parakeet, especially for technical vocabulary and accented speech. The on-device mode (available since macOS 13) avoids cloud processing but the quality gap is noticeable. Apple has not published benchmarks or model details.

Text-to-speech

Local TTS quality has improved substantially since early 2025. Three developments define the current moment.

Orpheus TTS (3B parameters) is the most expressive local TTS model available on Mac. Released by Canopy Labs and ported to MLX by the community, Orpheus generates speech with emotional range that approaches cloud leaders. It handles emphasis, pacing, and tonal variation convincingly. The tradeoff is size: 3B parameters requires approximately 1.88 GB of GPU memory, and generation is slower than smaller models.

Soprano (80M parameters) delivers fast multilingual TTS with eight voices. At 160 MB of GPU memory, it runs comfortably alongside other models. The quality sits between PocketTTS and Orpheus: natural-sounding speech without the emotional depth of the larger model, but generated in near-real-time.

PocketTTS runs on the Neural Engine via FluidAudio, producing speech with zero GPU impact. This architectural advantage matters: you can generate speech while running an LLM on the GPU without either workload slowing down. PocketTTS with the Cosette voice is the fastest TTS option, ideal for real-time voice chat loops where latency matters more than expressiveness.

Qwen3 TTS from Alibaba (0.6B parameters) introduced accessible voice cloning to local hardware. With a 10-30 second audio sample, it synthesizes speech in the cloned voice from arbitrary text. The multilingual support is broader than FluidAudio's curated voices, though English remains the highest-quality output language.

On the cloud side, ElevenLabs remains the quality leader for TTS and voice cloning, with 29 voices and industry-leading clone fidelity. Their pricing starts at $5/month with character limits. OpenAI's TTS API offers six voices at $15 per million characters. Amazon Polly provides solid quality at $4 per million characters. The cloud advantage is absolute quality ceiling; the local advantage is unlimited generation with zero per-character cost and complete privacy.

Voice cloning

Voice cloning crossed the usability threshold for local hardware in late 2025. Qwen3 TTS requires as little as 10-30 seconds of clear speech to produce a recognizable clone. Longer samples (1-2 minutes) improve quality. The clone captures pitch, cadence, and timbre well enough for practical use in content creation, accessibility (voice banking), and prototyping. ElevenLabs still produces higher-fidelity clones, especially for emotional range and accent reproduction, but the gap is narrowing with each model generation.

Voice is biometric data, as unique as a fingerprint. When you upload voice samples to a cloud cloning service, you hand biometric data to a third party with whatever retention policies they choose. Local voice cloning means the voice samples never leave your machine. The cloned voice model exists only on your hardware. For a technology with significant potential for misuse, local processing is not just a preference, it is a safeguard.

Streaming and real-time

Real-time streaming STT is now practical on local hardware. AudioPiper captures audio from any source on a Mac, including per-app capture via Core Audio Taps (macOS 14+), and streams it to FluidAudio's Parakeet model. Words appear within 1-2 seconds of being spoken. This enables live meeting transcription, real-time captioning, and streaming translation pipelines that run entirely on-device.

The distinction between batch and streaming transcription is not trivial. Batch processing gives you a transcript after the recording ends. Streaming processing gives you text while the conversation is still happening. The gap between "I will read the transcript later" and "I can see what was just said" changes how you participate in a conversation. You stop being the person furiously scribbling notes and start being the person who is actually present, with a live text record accumulating in the background.

Push-to-talk dictation via ActionPiper achieves approximately 140 milliseconds end-to-end latency from key release to text appearing at the cursor. This is faster than cloud dictation services (200-500ms minimum network overhead) and fast enough that text appears to materialize instantly. The FluidAudio STT backend stays loaded in memory as a keep-warm process, eliminating cold-start delays entirely.

The MLX Audio ecosystem

Apple's MLX framework has catalyzed a growing ecosystem of audio models optimized for Apple Silicon. The mlx-audio Python library supports multiple TTS model families: Kokoro, Spark TTS, Outetts, CSM, Dia, F5-TTS, Parler, and Bark, each with different strengths. Some excel at multilingual output, others at expressiveness or speed. ToolPiper bundles the same models through mlx-audio-swift, a native Swift port that runs without Python or pip. The model weights are identical; the interface is a Mac app instead of a terminal.

This ecosystem is why the local voice AI landscape is improving faster than any single model would suggest. Each new architecture that lands in MLX-Audio becomes available both as a Python library for developers and as a one-click install in ToolPiper for everyone else.

The architectural advantage: three processors, zero contention

This is the durable thesis for voice AI on Mac, and the reason local voice processing is not merely a privacy concession but an architectural superiority.

Apple Silicon has three independent processors capable of running voice models, and they do not compete for resources. The Neural Engine is a dedicated matrix-multiply accelerator designed for ML inference. The Metal GPU is a general-purpose parallel processor. The CPU handles orchestration and I/O. A full voice conversation pipeline — STT transcribing your speech, an LLM reasoning about what you said, TTS synthesizing a spoken response — runs all three models simultaneously on different hardware. No queuing. No context switching. No VRAM contention. Three workloads on three processors, all executing in parallel.

This architecture is why push-to-talk dictation achieves 140 milliseconds end-to-end latency. FluidAudio's Parakeet model sits loaded on the Neural Engine as a keep-warm process. When you release the key, inference begins immediately on dedicated silicon that was doing nothing else. The GPU might be running an LLM. The CPU might be handling I/O. Neither interferes.

Compare this to any other consumer platform. A PC with an NVIDIA RTX 4090 runs STT, LLM, and TTS on the same CUDA cores, contending for the same VRAM. Loading Whisper alongside a 7B LLM and a TTS model means all three compete for GPU memory and compute cycles. Running them simultaneously means time-slicing a single processor. The GPU is powerful, but it is one processor doing three jobs.

The practical consequence is that Apple Silicon can sustain a voice conversation loop at latencies that CUDA hardware cannot match, despite having lower peak throughput on any single model. An RTX 4090 transcribes faster than the Neural Engine in isolation. But when you need STT + LLM + TTS running concurrently — which is what a voice conversation actually requires — the Mac's three-processor architecture eliminates the resource contention that adds hundreds of milliseconds on single-GPU systems.

Unified memory amplifies this advantage. All three processors share the same physical memory pool with zero-copy data transfer. When Parakeet finishes transcribing on the Neural Engine, the resulting text is immediately available to the LLM on the GPU without a memory copy. When the LLM generates a response, it is immediately available to TTS without a transfer. On a discrete GPU system, data moves across the PCIe bus between CPU RAM and VRAM, adding latency at every pipeline boundary.

This is not a benchmarkable advantage in the traditional sense. No single-model benchmark captures it because the advantage only manifests when running a multi-model pipeline. It is an architectural property of the hardware, which means it is durable. It does not depend on any particular model being better or faster. It depends on the silicon layout of the chip, which does not change with a software update.

The memory constraint is real. Unified memory means all three processors draw from the same pool. A 16 GB Mac running Parakeet (2 GB on ANE), a 3B LLM (2-4 GB on GPU), and Soprano TTS (160 MB on GPU) uses roughly 4-6 GB for the voice pipeline alone, leaving room for the OS and other applications. An 8 GB Mac requires careful model selection. But this is a capacity constraint, not a contention constraint. The models share a memory pool; they do not share compute resources.

What's coming

Our roadmap

Streaming TTS improvements are in development for ToolPiper. Currently, TTS generates a complete audio buffer before playback begins. Streaming TTS will begin playback as soon as the first sentence is synthesized, reducing perceived latency for long text passages, especially in voice chat loops where response time shapes the conversational feel.

Speaker diarization for live transcription is planned. The current streaming pipeline produces a single text stream without identifying who spoke each segment. Adding speaker labels will make live meeting transcription more useful for multi-participant conversations where attributing statements to specific people matters.

Custom hotkey mapping for ActionPiper's push-to-talk is planned. The Right Option and Right Command keys are currently fixed assignments. Users with keyboard conflicts have requested configurable bindings.

Industry horizon

Multilingual TTS expansion is accelerating. Qwen3 TTS already supports a broader language set than English-focused models like Orpheus and Soprano. As MLX-Audio integrates more model architectures, the range of languages available for high-quality local synthesis will grow. Spark TTS, Dia, and F5-TTS are all available in the MLX ecosystem with varying language coverage.

Voice cloning from shorter samples is an active research area. Current models need 10-30 seconds of clean audio. Published research suggests that next-generation architectures may produce usable clones from as little as 3-5 seconds. No specific model has demonstrated this on local hardware yet, but the trajectory is clear from the progression: in 2024, cloning required minutes of audio. In 2025, 10-30 seconds became sufficient. The direction points toward single-digit seconds in the next generation.

Apple's on-device dictation continues to improve with each macOS release. Apple has not announced specific plans for enhanced local STT, but their investment in Neural Engine silicon and on-device ML frameworks suggests continued improvement. Any advances Apple makes to on-device speech recognition benefit the broader local voice AI ecosystem through hardware improvements.

How ToolPiper handles this today

ToolPiper bundles two audio backends and six voice-related templates, providing a complete local voice AI platform without external dependencies.

Two audio backends

FluidAudio runs on the Neural Engine via CoreML. It is a curated, zero-config backend that handles both STT (Parakeet V3) and TTS (PocketTTS with Cosette voice). FluidAudio is a keep-warm backend: the models stay loaded in memory for instant inference with no cold-start delay. It never receives unknown models; only tested, curated presets run on this backend. This design means FluidAudio is predictable and fast, always ready, always the same quality.

MLX Audio runs on the Metal GPU. It is a general-purpose audio backend that accepts both curated presets (Soprano, Orpheus, Qwen3 TTS) and unknown models from HuggingFace. It is the category default for all audio workloads, meaning any new TTS model you download routes through MLX Audio automatically. Higher-quality voices with more natural prosody, at the cost of GPU memory and slightly higher latency.

The two-backend architecture is intentional. FluidAudio on the Neural Engine handles the latency-sensitive work: real-time dictation, push-to-talk, streaming transcription. MLX Audio on the GPU handles the quality-sensitive work: expressive narration, voice cloning, content creation. You can run both simultaneously because they use different hardware.

Templates

Voice Input. Microphone or audio file in, text out. Uses FluidAudio STT with Parakeet V3 on the Neural Engine. The starting point for any transcription workflow. Accepts MP3, WAV, M4A, AAC, and FLAC formats. Ready to try it? Set up voice transcription — takes about 60 seconds.

Text to Speech. Text in, natural speech out. Choose from PocketTTS (Cosette, Neural Engine), Soprano (Tara default, 8 voices, GPU), or Orpheus (Tara default, 8 voices, GPU). All voices default to female per our voice policy. Audio plays back inline with waveform visualization and can be downloaded as a file for use in other workflows. Ready to try it? Generate speech from text.

Voice Chat. Full spoken conversation with AI. Chains Audio Capture, STT, LLM, and TTS into a voice loop. Speak naturally, hear the AI respond. Each stage is independently swappable: change the LLM for different reasoning quality, swap the TTS voice for different character, or switch STT models if you need different language coverage. The visual pipeline builder shows you exactly what's happening at each stage. Ready to try it? Start a voice conversation.

Transcribe & Summarize. Audio in, structured key points out. Two models chained: STT transcribes, then an LLM extracts decisions, action items, and key points. Customize the summary format by editing the LLM's system prompt: meeting minutes, key decisions only, client-ready executive summary, or technical review. Both the full transcript and the summary are preserved. Ready to try it? Summarize a meeting recording.

Live Translate. Speak in one language, hear the translation in another. Chains STT, LLM (translation), and TTS into a real-time loop. Source language detection is automatic via Parakeet's multi-language support. Edit the system prompt to set the target language. Modern LLMs handle idioms and context better than traditional machine translation because they understand meaning, not just word-for-word substitution. Ready to try it? Build a translation pipeline.

Voice Clone. Short reference audio plus text equals speech in the cloned voice. Qwen3 TTS handles voice encoding and synthesis locally on the GPU. As little as 10-30 seconds of clear speech required. The template shows two input blocks side by side: one for your audio sample, one for the text you want spoken. Ready to try it? Clone a voice on your Mac.

Push-to-talk (ActionPiper)

ActionPiper is a free menu bar app that registers two global hotkeys for voice input, requiring roughly 20 MB of memory. Right Option = push-to-talk dictation: hold, speak, release, and transcribed text appears at your cursor in any application. Your IDE, browser, Slack, Notes, Terminal, anywhere that accepts text input. No app switching, no mode changes, no clipboard involved. Right Command = push-to-command: hold, speak an instruction in natural language, release, and your Mac executes it via ActionPiper's 26 action domains covering display, audio, windows, apps, network, Bluetooth, media controls, accessibility, and more. The LLM interprets natural language, so "make it dark" and "turn on dark mode" both work. Both modes use FluidAudio STT on the Neural Engine as a keep-warm backend with approximately 140ms end-to-end latency. Ready to try it? Set up push-to-talk.

AudioPiper integration

AudioPiper captures audio from any source on your Mac: microphone, system audio, or individual apps via Core Audio Taps (macOS 14+). It streams mixed PCM audio over WebSocket to FluidAudio for real-time transcription. Per-app capture means you can transcribe Zoom audio without picking up Spotify in the background. Capture only the browser tab playing a webinar. Combine microphone input with system audio for a complete record of both sides of a call. No virtual audio drivers, no kernel extensions, no third-party audio routing software that breaks with every macOS update. Ready to try it? Try live transcription.

MCP tools

ToolPiper exposes voice capabilities as MCP tools: transcribe (audio file to text), speak (text to audio), and chat (with voice pipeline integration). Any MCP client, including Claude Code, Cursor, and Windsurf, can invoke local voice AI programmatically. This means AI coding assistants can generate speech, transcribe recordings, or run voice workflows as part of automated pipelines without any manual intervention.

Models and hardware

Every model listed below runs entirely on your Mac's Apple Silicon. No cloud processing, no API keys, no per-use pricing. The hardware column indicates which processor handles inference, which matters for understanding resource contention when running multiple models simultaneously.

Apple Silicon's unified memory architecture means both the Neural Engine and the Metal GPU share the same memory pool. A model loaded on the Neural Engine and a model loaded on the GPU both consume from your Mac's total RAM. The advantage is zero-copy data transfer between processors; the constraint is that total memory usage across all loaded models must fit within your available RAM.

For voice chat (STT + LLM + TTS running simultaneously), budget approximately 2 GB for STT (Parakeet on ANE), 2-4 GB for a 3B LLM (on GPU), and 160 MB to 1.88 GB for TTS depending on voice quality. A 16 GB Mac handles this comfortably. An 8 GB Mac requires smaller models and tradeoffs.

How does this compare to cloud?

The honest comparison between local voice AI and cloud services comes down to five dimensions: privacy, cost, quality, speed, and convenience.

Privacy is the clearest advantage of local processing. Cloud transcription and TTS services process your audio and text on their servers. Otter.ai's terms give them rights to use data for product improvement. Google's Speech API logs requests by default. ElevenLabs stores voice samples for cloning on their infrastructure. Local processing means your audio, text, and voice biometric data never leave your machine. For meeting recordings with confidential content, legal discussions, medical dictation, or HR conversations, this is not a theoretical benefit.

Cost favors local processing for regular use. Otter.ai Pro costs $16.99/month. ElevenLabs starts at $5/month with character limits. Google Speech API charges $0.006-0.024 per minute. Amazon Polly charges $4 per million characters. ToolPiper is free with unlimited usage for all voice capabilities. If you transcribe meetings daily or generate substantial TTS output, the cost savings accumulate quickly. The tradeoff is hardware: you need an Apple Silicon Mac, which is a sunk cost if you already have one.

Quality is where cloud services still hold an advantage at the top end. ElevenLabs produces the most natural TTS and the highest-fidelity voice clones. Google's speech recognition handles 125+ languages versus Parakeet's 25. Otter.ai offers speaker diarization that local tools don't yet match. But for clear speech in supported languages, Parakeet V3's accuracy is comparable to cloud STT. And for everyday TTS use, Orpheus and Soprano produce voices that most listeners cannot distinguish from mid-tier cloud options.

Speed is nuanced. For batch transcription, Parakeet V3 at 210x realtime is faster than any cloud service because there is no upload time. For push-to-talk dictation, 140ms local latency beats cloud services with their 200-500ms network overhead. For TTS, local and cloud are comparable. For voice chat, local avoids the 500ms-2s network round trip that cloud services impose.

Convenience varies by use case. Cloud services require accounts, API keys, and subscriptions. ToolPiper requires one app install. Cloud services work on any device. ToolPiper requires a Mac with Apple Silicon. Cloud services handle more languages. ToolPiper works offline, on a plane, in a location with no connectivity.