If you have an hour of audio you genuinely don’t want a third party reading, a confidential interview, a therapy session, a deposition, a doctor’s dictation, a board meeting, the standard advice (“just upload it to a transcription service”) is exactly the wrong move. The moment that file leaves your machine, you’ve handed a copy of a sensitive conversation to a company whose retention policy you didn’t read and can’t enforce. Local transcription solves this at the root: the audio is processed entirely on your own computer, and nothing is ever uploaded. OpenAI’s Whisper models are open weights, run offline, and produce transcripts that are genuinely close to commercial cloud quality.
This guide is the practical version. Which Whisper model to use, which tool to run it with, how fast it’ll actually go on your hardware, and where the honest limits are. No hype, just enough to get a clean transcript of sensitive material without it ever touching the internet.
Why local, not cloud, for sensitive audio
The privacy argument for transcribe audio locally isn’t abstract. Cloud transcription means your recording is transmitted, stored (often for some retention window), and processed on infrastructure you don’t control. For most cat videos, who cares. For the categories of audio people actually agonize over, patient information, privileged legal discussion, journalistic source material, HR investigations, that data flow is the whole problem. This is the same calculus that makes running AI locally attractive in the first place: privacy by construction, not by promise.
There’s also a compliance angle. If you handle audio under confidentiality obligations, the kind of material a lawyer would treat as privileged, keeping the file on a machine you control sidesteps an entire category of “did we just send protected data to a vendor?” questions. We get into this specifically in local AI for confidential legal work, and the logic carries straight over to transcription. The general case for owning the pipeline rather than renting it is laid out in local AI vs cloud AI.
The good news: transcription is one of the easiest local AI workloads to run well. Speech-to-text models are far smaller and lighter than chat LLMs, so even modest hardware gets usable results.
The Whisper model family: tiny to large-v3, plus turbo
Whisper isn’t one model, it’s a family of sizes, all using the same architecture, trading accuracy for speed and memory. Bigger models are more accurate (especially on accents, noise, and non-English audio) but slower and hungrier for memory. As a rough mental model, treat these as bands, not exact specs:
| Model | Relative accuracy | Relative speed | Memory footprint | Good for |
|---|---|---|---|---|
| tiny | lowest | fastest | smallest (well under ~1GB) | quick drafts, clean English |
| base | low | very fast | small | clean audio, fast turnaround |
| small | good | fast | small-moderate | the practical sweet spot for many |
| medium | better | moderate | moderate | accents, some noise |
| large-v3 | best | slowest | largest | hard audio, non-English, accuracy-critical |
| large-v3-turbo | near-large | much faster than large | lighter than large | the modern default for most people |
A few honest notes on this table:
- large-v3-turbo is the one to reach for first. Released in late 2024, turbo is a distilled variant of large-v3, its decoder was cut from 32 layers down to a handful, dropping it from ~1.5B to roughly 0.8B parameters. The result is dramatically faster decoding (commonly cited in the several-times-faster range versus full large-v3) with only minor accuracy loss, mostly negligible for English. For sensitive work where you want strong accuracy but don’t want to wait all day, turbo is usually the right call.
- distil-whisper is a separate distilled line aimed at speed. Variants like distil-large are reported to run several times faster than large-v3 while staying within roughly a percentage point of its word error rate on long-form English. If your audio is English and you want maximum throughput, it’s worth a look, but it’s more English-focused than the multilingual checkpoints.
- As of mid-2026 there’s no official Whisper v4. large-v3 and large-v3-turbo remain the current production-grade open checkpoints, so you’re not missing a newer model.
If you’re brand new to any of this, local AI for beginners covers the basic concepts (models, weights, quantization) that the rest of this assumes.
The tools: same model, very different engines
Here’s the part that trips people up. Whisper the model and Whisper the program you run are different things. There are several implementations that load the same weights but behave very differently on speed, memory, and which hardware they like. The three that matter:
| Tool | What it is | Best on | Why pick it |
|---|---|---|---|
| whisper.cpp | Pure C/C++ port, no Python | Apple Silicon, low-resource, “just works” | No CUDA toolkit, no Python env; runs almost anywhere |
| faster-whisper | CTranslate2 reimplementation | NVIDIA GPUs (and solid CPU) | Big speedups via INT8/FP16; production-grade throughput |
| OpenAI Whisper (reference) | The original PyTorch code | Anywhere with PyTorch | The reference; usually the slowest of the three |
whisper.cpp, the no-dependencies path
whisper.cpp is a from-scratch C/C++ implementation. Its selling point is that it has essentially no dependency baggage: no Python environment to wrangle, no CUDA toolkit to match versions with. It compiles to a small binary, runs on Linux, Windows, and especially well on Apple Silicon Macs (where it uses the GPU via Metal). If you’ve ever lost an afternoon to a broken Python/CUDA setup, this is the antidote. Models are distributed as .bin (GGML/GGUF-style) files you download once and point the binary at.
Typical shape of use: build it, grab a model file, run something like ./main -m models/ggml-large-v3-turbo.bin -f audio.wav. (Check the project’s current README for exact flags, they evolve.)
faster-whisper, the speed path on NVIDIA
faster-whisper is built on CTranslate2 and is the one to use when you have an NVIDIA GPU and care about throughput. It uses INT8/FP16 quantization and optimized kernels to run several times faster than the reference implementation on GPU, and meaningfully faster on CPU too, while keeping accuracy effectively the same. It’s a Python library, so you’ll need a working Python environment, but for batch-transcribing hours of audio it’s hard to beat. The turbo model in INT8 is notably light on VRAM, small enough that it fits comfortably even on modest cards.
Easy GUI options (no terminal required)
If you’d rather not touch a command line, several desktop apps wrap these engines:
- Buzz, cross-platform desktop app, imports audio/video and exports transcripts and subtitles.
- MacWhisper, a polished Mac-native front end (built on whisper.cpp).
- Vibe, a cross-platform GUI that runs Whisper locally with a simple drag-and-drop flow.
- Subtitle Edit, long-standing subtitle tool that can drive Whisper for transcription/captioning.
- WhisperX, more of a power tool: adds word-level timestamps and speaker diarization (who-said-what) on top of Whisper.
These all keep processing on-device; a GUI doesn’t change the privacy story, only the convenience. Pick whichever matches your OS and comfort level.
CPU vs GPU: how fast will it actually go?
This is the question everyone asks, and the honest answer is “it depends, here are the bands.”
- On a decent NVIDIA GPU, transcription is comfortably faster than real time, often by a large multiple with faster-whisper and a turbo/distil model. An hour of audio can finish in a small fraction of that. Treat “many times real time” as the ballpark, not a guarantee; it swings with model size, card, and audio length.
- On Apple Silicon, whisper.cpp with Metal is genuinely good, many users get faster-than-real-time on small/medium models, with large being slower but still practical.
- On CPU only, it still works, this is one of the workloads that’s perfectly viable without a GPU. Smaller models (tiny/base/small) can run near or faster than real time on a modern multi-core CPU; large-class models on CPU are much slower than real time and better suited to “start it, go make coffee” batch jobs. If you’re GPU-less, see running local AI without a GPU for the general playbook, transcription is one of the friendliest tasks for that setup.
The practical takeaway: model size is your main speed/accuracy/memory dial. Start with large-v3-turbo. If it’s too slow on your hardware, drop to small or medium. If accuracy on hard audio matters more than time, step up to full large-v3 and accept the wait. If you’re spec’ing a machine specifically for this kind of work, the best budget AI PC build walks through where the money actually matters.
Languages: it’s not English-only
Whisper’s multilingual checkpoints handle on the order of ~100 languages, with quality that varies a lot by language, strong on widely-spoken, well-resourced languages and weaker on low-resource ones. It can also auto-detect the spoken language and even translate speech into English. Two caveats worth stating plainly:
- distil-whisper is more English-centric than the standard multilingual models. For non-English audio, prefer large-v3 or large-v3-turbo.
- Accuracy degrades with bad audio. Heavy background noise, crosstalk, strong accents, and low-bitrate phone recordings all push word error rate up regardless of model. Garbage in, garbage out still applies.
Honest limitations
Local Whisper is excellent, but it’s not magic. Set expectations:
- Speaker labels aren’t built in. Vanilla Whisper gives you text, not “who said what.” For diarization you need an add-on like WhisperX, and even then it’s imperfect on overlapping speech.
- It hallucinates on silence and noise. Whisper can occasionally invent text during long pauses or non-speech segments. Skim the output; don’t trust a sensitive transcript blindly.
- Exact numbers vary. Anyone quoting you a precise word-error-rate or a guaranteed tokens-per-second figure is overselling. Real performance depends on your audio, model, and hardware, benchmark on your files.
- Long files want chunking. Most tools handle this for you, but very long recordings transcribe more reliably when processed in segments.
The privacy bottom line
The entire reason to do this is that the recording never leaves your computer, no upload, no third-party retention, no terms-of-service lottery over who can access a private conversation. That’s a categorical difference from cloud transcription, not a marginal one, and it’s the same reason people get nervous about what their employer can see in their chat history: once data leaves your control, you’re trusting a policy instead of a fact. With local transcription, the privacy guarantee is structural, verifiable by the simple act of pulling your network cable and watching it still work.
Start with large-v3-turbo on whisper.cpp (if you want zero-setup) or faster-whisper (if you have an NVIDIA GPU and lots of audio). Drop to a smaller model if you need more speed, step up to full large-v3 if the audio is hard. Either way, your sensitive recordings stay exactly where they belong: on a machine you own.