Teaching Whisper your jargon: initial prompt and replacements
Whisper is good at ordinary speech and bad at your project's names. It has never seen your internal service or your teammate's surname, so it writes the closest common word. There are two fixes. The initial prompt nudges the model toward your spelling before it decodes, and replacements correct the text afterwards. FnFlow uses both, and here's how each one works.
What the initial prompt does
Whisper writes text one token at a time, and each token depends on the text that came before. The initial prompt fills that "text before" with words you choose, as if they had just been said. OpenAI's own Python package describes it as context for the first window, for example custom vocabularies or proper nouns, to make those words more likely.
It is not an instruction. OpenAI's speech-to-text guide says Whisper doesn't follow instructions the way a general-purpose text model does. Its prompting cookbook adds that Whisper copies the style of the prompt, capitalization and punctuation included, and that a glossary of correct spellings fixed misspelled names in its examples. A prompt also can't change what was said. It only shifts the odds between spellings that fit the audio.
Using it in whisper.cpp
With the command-line tool, pass the terms with --prompt:
whisper-cli -m models/ggml-large-v3-turbo-q8_0.bin -f memo.wav -l auto \
--prompt "Supabase, Postgres, pgvector, Zustand"
whisper-server accepts the same thing as a form field on each request:
curl 127.0.0.1:8080/inference -F file=@memo.wav -F prompt="Supabase, Postgres, pgvector"
The help text gives the limit as half the model's text context. In whisper.cpp that context is 448 tokens, so the prompt gets 224, the same figure OpenAI publishes. If you go over, the code keeps the last tokens and drops the start, so put the terms you care about most at the end.
Whisper reads audio in 30-second windows. The initial prompt is used for the first one, and later windows are conditioned on the text already decoded. For a long recording, --carry-initial-prompt prepends your prompt to every window. A typical dictation fits in one window, so this rarely matters there.
Replacements catch what the prompt misses
A prompt makes a spelling more likely, and a replacement makes it certain. It's a plain find-and-replace on the finished text. FnFlow applies replacements to whole words, ignores case and runs longer entries before shorter ones, so "kubernetes" becomes "Kubernetes" without touching a word that merely contains it. Replacements work on the output of either engine, including Apple's on macOS 26.
What we measured
We tried five terms on short test sentences. With large-v3-turbo, "update Zapot" became "update Saypad". The small model heard "post-GIRS" without the prompt and wrote "Postgres" with it. We also sent the same audio file to whisper-server twice. Without the prompt it returned "Zapot", and with the prompt it returned "Saypad".
It wasn't perfect. A Russian sentence came back with "Postgres" right and "Supabase" spelled "Supabaes", which is the kind of near miss a replacement fixes.
Side effects to watch
Names can switch alphabet. When a Russian name was in the vocabulary in Cyrillic, the model sometimes wrote it in Latin letters.
The prompt can echo on silence. With no speech to decode, Whisper may hand your prompt back as the transcript. FnFlow treats a result identical to the prompt as empty, and trims a copy of the whole prompt from the start of a result when it has three or more terms. A single vocabulary word at the start of a sentence stays, because you might have said it.
Style carries over too. Since Whisper copies the prompt's style, write each term exactly as you want it to appear.
FnFlow's vocabulary
Settings → Vocabulary takes up to 60 terms, with duplicates removed regardless of case. The list goes to Whisper as the prompt on every dictation, and an empty list clears it. Replacements live in the same place. Everything is stored in vocabulary.json in FnFlow's folder, readable only by your user. To check a new term, run an older recording that got it wrong through recognition again from the history. If you'd like to try the prompt with plain whisper.cpp first, running Whisper locally on a Mac covers the setup, and how FnFlow was built explains where the vocabulary fits in.
Questions
How long can a Whisper prompt be?
224 tokens. whisper.cpp's help lists the limit as half the model's text context, and OpenAI's docs give the same number. Tokens aren't words, and unusual names often take several tokens each. If the prompt is longer, only the end is used.
Can I tell Whisper to always spell a word a certain way?
Not with an instruction. OpenAI says Whisper doesn't follow instructions like a chat model. Write the word in the prompt with the spelling you want, and add a replacement if it still comes out wrong.
Does the prompt work for languages other than English?
In our tests it helped with Russian speech full of English product names, though not perfectly. One term came out as "Supabaes", which a replacement then fixed.
Does FnFlow send my vocabulary anywhere?
No. It's stored in a local file and passed to Whisper running on your Mac, through a server that listens on 127.0.0.1 only.
Local Wispr Flow alternativeFnFlow vs VoiceInkVoice input for CursorHow 7 Mac dictation apps get you to your first dictationWhisper.cpp on Apple Silicon: which large-v3-turbo quantization to shipHow I built a local dictation appVoice dictation in Claude Code and the terminalWhy holding fn doesn't work for dictation on Mac (and how to fix it)How to run Whisper locally on an Apple Silicon MacSpokenly alternative for developersFnFlow vs EnviousWisprHow to dictate a good prompt for AI coding toolsVoice dictation in VS Code on a MacWhy Whisper types "Thank you" on silence, and how to filter itPrivate dictation on a Mac at work: what actually leaves your machineVoice input for Codex CLI and the ChatGPT appVoice coding on a Mac: what actually worksWriting git commit messages by voiceDictating code review comments on GitHubDictation in IntelliJ IDEA, PyCharm and WebStorm on a MacDictation in Zed on a MacDictation in Terminal, iTerm2 and WarpWhich Whisper model to use on an Apple Silicon MacPush-to-talk dictation on a Mac: hold, toggle or streamSuperwhisper alternative for developersClaude Code voice mode: how to enable /voice and fix itWhisper apps: five ways to run Whisper locallyTalon voice coding vs a dictation appPush to talk on a MacDoes Claude have a voice mode?VoiceInk vs Wispr FlowVoiceInk vs SuperwhisperParakeet vs Whisper on a MacApple SpeechAnalyzer vs WhisperWhisper.cpp vs faster-whisper on a MacMlx-whisper vs whisper.cppGitHub Copilot voice inputGemini CLI voice inputVoice input in Google AntigravityVibe coding with voice input