Updated October 2026: added the Dictation API (launched September 15), a sync vs streaming decision guide, a keypress-to-text latency budget, and guidance for clinical dictation and PHI.
On August 17, the leading dictation app closed a $280M Series B at a $2B valuation. Whatever anyone believed about voice input as a category last year, the demand question is settled. It got settled by a balance sheet rather than by an argument.
The how isn't settled at all.
Ask ten teams shipping voice input what a dictation feature consists of and you'll get ten different lists, most of them assembled after launch in response to user complaints. Nearly all start from the same assumption: that dictation is a transcription call with a microphone in front of it. It isn't. Teams who find that out in week six rebuild what they shipped in week two.
So this is the inventory, written from the build side. It's not a roundup of apps to install, and it's not an explainer on what a dictation API is — that page covers the mechanics. What follows is the feature list your users will measure you against, whether you're adding voice to a notes app, a collaboration tool, a developer tool, or a clinical workflow.
Seven features. Two architectures. A latency budget. And one thing nobody puts on the list.
The seven features that define a dictation feature in 2026
Dictation used to be a toggle. Now it's a system, and users notice each piece of it individually, usually by noticing an absence.
| Feature | What users expect | Where it belongs in the stack |
|---|---|---|
| AI cleanup | Filler sounds, false starts, and stammered repeats gone: the text they meant, not the words they said | Rewrite pass, after recognition |
| Custom vocabulary | Drug names, package paths, ticket IDs, and internal product names come back right | Recognition layer: keyterms plus contextual prompting |
| Push-to-talk and hands-free | Hold a key for a short burst, double-tap to stay open, and remap either one | Client capture layer |
| Voice commands and spoken punctuation | "New paragraph," "delete that," and spoken parentheses rendered as characters | Command handling inside the rewrite pass |
| Context awareness | The same sentence formatted for a terminal, an email, or a half-written paragraph | Context string sent with the rewrite request |
| Dictation history | Nothing lost to a network blip or a crash, searchable, with clear retention | Application storage |
| Predictable latency | Text on screen before they look up: a p50 you can animate around and a p95 that holds | Transport and model choice: a sync endpoint built for bounded clips |
AI cleanup (the one that changed the category)
Speech recognition returns what was said. People want what they meant.
Those are different products. Read a verbatim transcript of yourself talking and you'll find what cleanup removes: filler sounds, a false start, a sentence you abandoned halfway, and a word you said twice. None of that is a recognition error. It's a faithful record of how humans talk, and it's not text anyone wants to paste into a Jira ticket.
Cleanup is what separates 2026 dictation from the dictation built into your operating system, and users feel it in the first ten seconds. Building it yourself means a second inference pass, a cleanup prompt you'll rewrite a dozen times, and rules specific enough to drop a stammered repeat without eating a real word. Our deep dive on how dictation cleanup works covers the rules that hold up.
Custom vocabulary and jargon handling
This is the most repeated complaint in every community thread about dictation, in every vertical.
A developer dictates a package path and gets "bolt module pa portal" instead of bolt.module.paportal. A clinician who spent three months training a Dragon vocabulary still watches "thickened endometrium" come back as "chicken endometrium." The words that matter most are the ones a general model has least reason to expect: internal product names, drug names, version strings, ticket IDs, surnames.
The fix belongs at the recognition layer, not the cleanup layer. It's cheap in the right order: keyterms prompting for terms you know are coming, plus contextual prompting to describe the setting the audio comes from. And track entity accuracy rather than word accuracy, because aggregate word error rate will happily hide every miss your users care about.
Push-to-talk and hands-free modes
Two interaction models, and shipping one of them annoys half your users.
Hold a hotkey and speak, release and the text lands: that's push-to-talk, and it's what people want for short bursts inside another app. Double-tap the same key and stay open, and that's hands-free, for the long note where holding a key gets tiring. Ship both, plus configurable shortcuts, because a hotkey that collides with someone's IDE binding is a hotkey they turn off.
Our walkthrough on how to build push-to-talk dictation covers capture, encoding, and the error cases real hardware produces.
Voice commands and spoken punctuation
"New paragraph." "Delete that." "Send it."
Voice commands look like polish and turn out to be load-bearing. The clearest case is medicine, where it isn't a preference at all: physicians are trained to speak their punctuation. A doctor says "patient open parentheses Andrew close parentheses is tired" and expects (Andrew) on the screen, because that's how they've dictated into a recorder for years.
Get it wrong and you don't get a bug report. You get a clinician who hand-edits every note and quietly stops using the feature. That's one reason clinical dictation carries its own expectations instead of inheriting the consumer ones.
Context awareness
The same sentence should come out differently depending on where it lands.
Dictated into a terminal, "run the tests and fix whatever breaks" is an instruction, which is the whole premise of dictating to coding agents. Dictated into an email, it's a request that needs a greeting. Dictated into a field that already holds half a paragraph, it needs to continue that paragraph's tense and register. Context means what app you're in, what's already in the field, and what the user had selected when they started talking.
It's the least visible feature here and the biggest quality jump per line of code. It costs one extra string in the request and changes every output.
Dictation history
Boring. Expected. The loudest complaint when it's missing.
Users treat dictation as a place they put thoughts, so they assume the thoughts are kept. A dropped transcript (a network blip during a four-minute brain dump, a crash, a window that closed) is the first reported failure of thin voice integrations. It turns an enthusiastic user into a suspicious one permanently. Ship a history tab, make it searchable, and be explicit about what you retain. Voice-note transcription features hit the same wall: "where did my recording go" is the support ticket that never stops.
Latency you can design around
The bar isn't "fast." It's predictable.
Dictation is a UI-blocking interaction. The user has stopped talking, they're looking at the cursor, and there's no background job to hide behind. What matters is the shape of the distribution, not your best result out of twenty attempts: a p50 you can design an animation around, plus a p95 that doesn't embarrass you.
That's why short-clip endpoints exist as their own product class. The Sync API returns a complete transcript for a bounded clip at roughly 134 ms p50, and dictation is the first use case named on its page. That's not a coincidence; it's the workload that motivated it.
Granola, whose notes app runs on AssemblyAI transcription, put the user-visible version of that better than a chart can:
“The speed difference is immediately noticeable — our users see their conversations transcribed almost instantaneously. It feels so much more responsive than what we were using before.”
— Jonathan Kim, Software Engineer, Granola
Send a bounded clip and get a transcript back fast enough to drop straight into the field.
Sync, streaming, or async: which API fits dictation?
Most dictation builds go wrong at the transport choice, before anyone writes a cleanup prompt. Here's how the three shapes compare for a short burst of speech.
| API shape | How it works | Fit for dictation |
|---|---|---|
| Async (pre-recorded) | Upload a file, poll or wait for a webhook | Poor. There's a floor of several seconds you can't engineer under, and nobody waits five seconds for a sentence. |
| Streaming (WebSocket) | Hold a socket open and receive partial and final transcripts | Good for long, open-ended sessions. Costly for repeated short bursts, because each burst pays a fresh connection setup of a couple hundred milliseconds. |
| Sync (one HTTP request) | POST a bounded clip, get the finished transcript in the response | Best fit. No upload step, no polling, no session to manage. It accepts 80 ms to 2 minutes of audio per request. |
Most dictation bursts run 10 to 12 seconds. That's exactly the bounded clip Sync was built for. Streaming still wins when the user dictates for minutes and wants to watch words appear, which is the chunk-per-pause pattern below. For a full comparison of all three shapes, see real-time vs batch transcription.
There's now a fourth option: skip the stitching. The Dictation API, launched September 15, 2026, takes one clip and returns two things in one call: the verbatim transcript in text and finished, send-ready text in llm_response. Cleanup runs by default. Pass llm_instruction to reshape the output as a task list, a house style, or a SOAP note. It runs at Universal-3.5 Pro accuracy across 19 languages for $0.62/hr all in.
So the decision comes down to who owns the cleanup prompt:
- Sync API plus your own rewrite pass ($0.30/hr plus LLM tokens) if cleanup is your product surface and you want full control.
- Dictation API ($0.62/hr all in) if you want finished text in one call and want to ship this sprint.
The keypress-to-text latency budget
"Under a second" is the number that matters. In the field, users accept text that appears within a second of letting go of the key. Past that, they start watching the cursor. Here's where that second goes for a short clip, as measured in our September 2026 dictation workshop.
| Stage | Typical | What moves it |
|---|---|---|
| Speech-to-text inference | ~134 ms p50 (Sync API) | Shorter clips, trimmed silence |
| Transport | ~100 ms when the connection is pre-warmed; more when cold or far from the region | Connection pre-warming, staying in region |
| Cleanup pass | ~500 ms with a small, fast model | Smaller model, shorter prompt |
| Total | Under one second |
Two moves do most of the work. First, warm the connection when the user presses the key, not on page load. The TLS handshake then runs while they're still talking, which is dead time you were going to spend anyway. Second, keep the cleanup model small. Qwen3.5 4B Fast on LLM Gateway is hosted by AssemblyAI for exactly this shape of work. Every upgrade to a bigger model trades milliseconds for intelligence, and in dictation the milliseconds are the product.
Long rambles cost more. A one-line Slack reply is the fast case, and it's also the common case.
Two patterns teams are using today
Once you have the feature list and the transport, the architecture question narrows. Nearly everything shipping today is one of two shapes.
One clip per utterance
The user controls the start and the stop. They hold a key, speak, and release, and you send one bounded clip and get one finished result back.
This is the shape that fits dictation, because the audio is already complete when you send it. There's no turn to detect, no endpointing heuristic to tune, and no partial hypothesis to render and then revise on screen.
One request also fails more gracefully than a socket does. A single POST survives a WiFi-to-cellular handoff that would kill a WebSocket mid-sentence.
Chunk per pause
The second pattern cuts on natural silence and inserts progressively instead of dumping a block of text at the end.
Teams reach for it when dictation runs long, which in practice means clinical documentation. A routine SOAP note runs 250 to 400 words, or two to three minutes of speech. Complex hospitalist and psychiatric notes run past 800. Nobody dictating that long wants to watch a blank field for three minutes and then read a wall of text.
Chunking on pause solves the experience problem and the architectural one together. Each request stays inside the per-clip ceiling (2 minutes on Sync), and the person dictating gets something to react to while they're still talking. The tradeoff is that your cleanup layer sees each chunk without its neighbors, so context has to be carried forward deliberately. For continuous live dictation, a streaming connection on Universal-3.6 Pro Realtime is the other option.
Dictation for clinical workflows and PHI
Clinical dictation is where every feature on the list gets stress-tested at once: spoken punctuation, drug names, long notes, and protected health information.
Three things change when the audio contains PHI:
- Sign a BAA first. AssemblyAI is considered a business associate under HIPAA and signs a Business Associate Addendum (BAA) for customers processing PHI. Paid accounts can sign it self-serve. Confirm which endpoints your BAA covers before sending PHI through the Sync or Dictation APIs.
- Get drug names right at recognition. Pass a prompt describing the setting ("a physician dictating a progress note") and keyterms for the drugs your users prescribe. Cleanup can't recover a drug name that was misheard.
- Treat the rewrite as formatting, not documentation. An llm_instruction can shape dictation into a SOAP layout. The clinician still reviews and signs it.
For longer clinical recordings that need Medical Mode, use pre-recorded or realtime speech-to-text rather than Sync. The BAA FAQ covers signing.
Who's shipping what
The list above isn't theoretical. It describes what's already in market.
Fireflies ships a desktop dictation app where Fn dictates into any application. A double-tap switches to hands-free, shortcuts are configurable, there's a dedicated dictation history tab, and they're explicit about not logging private data. Every one of those maps to an item above. Granola offers dictation on desktop and mobile, and exposes model choice on mobile, a strong signal about how much users care about the cleanup layer once they've compared two of them. Confluence and Rovo are adding dictation-style input to a surface nobody was asking to talk to two years ago.
And if you want to feel the full loop before writing code, Blurt is a free, MIT-licensed Mac dictation app built on the Dictation API. Hold a key, talk, and the text lands in whatever app you're in.
Different products, converging spec. When several teams independently ship the same seven things, that's not a trend. It's a baseline.
What the build looks like today
Here's the part teams underestimate, because on a whiteboard it's two boxes.
You capture bounded audio, you transcribe it, and you run the transcript through a rewrite pass. The transcription half is the easy half. The Sync API is built for exactly this shape: audio that's already complete, and a person waiting on the text. It runs on Universal-3.5 Pro, the same flagship model behind our pre-recorded endpoint, not a smaller variant tuned for speed. That matters more than it sounds like it should. In most stacks, "make it faster" means "use the smaller model," and the accuracy tax hides inside the latency win.
The rewrite half is where the sprint goes.
You'll wire the transcript to a model, and then discover the failure modes nobody scopes. The model will occasionally answer a question it heard in the audio instead of writing it down. It will rewrite input that didn't need rewriting, expanding a clean three-word utterance into something more formal than the person who said it. You'll need to detect timeouts on the second hop and decide what to render when it fails: the raw transcript, an error, or a spinner nobody wants.
And the prompt looks like a paragraph and behaves like a product surface. The version that works doesn't say "remove filler words." It specifies handling rule by rule, plus an explicit list of content words the model must never drop. Getting a rewrite to remove things is easy. Getting it to stop removing things is the hard part.
That's the work the Dictation API absorbs. It returns the verbatim transcript and the rewrite side by side, falls back to the transcript if the rewrite times out, and never executes a dictated command. If cleanup is core to your product, build it yourself on Sync. If it isn't, don't spend the sprint.
Now the boundary, because it's the thing teams most often assume works the other way.
The rewrite polishes. It does not correct. Hand it a misheard drug name and you get a confidently formatted wrong drug name, properly capitalized and correctly punctuated. That's worse than the messy version, because it reads authoritative. The fix is upstream work, at the recognition layer where the word was lost.
Send a messy dictated clip and see what the transcript gives you before any cleanup runs.
The feature nobody lists
Every feature above is table stakes, and none of them is what users judge you on.
What they judge is whether the text is there when they look up from the microphone. That's the whole evaluation, and it happens before they've read a word of the output. Cleanup quality gets noticed on the second use. Jargon accuracy gets noticed on the fifth. Whether the text beat their eyes back to the screen gets noticed every time, and it sets the mood for everything they read next.
Which says something slightly uncomfortable about how these features get prioritized. Cleanup, vocabulary, history, and commands go on a changelog. "There when they look" doesn't, and it decides whether anyone uses the rest. Put the latency budget on the roadmap as a feature, with an owner.
Bounded clips, flagship accuracy, and finished text back fast enough to drop into the field.
Frequently asked questions
What features should a dictation app have?
A modern dictation app ships seven things: AI cleanup, custom vocabulary, push-to-talk and hands-free modes, voice commands including spoken punctuation, context awareness, searchable history, and predictable low latency. Users notice each one individually, and they notice absences faster than features.
How does AI dictation cleanup work?
AI dictation cleanup takes a verbatim transcript and returns the text the speaker meant. It removes filler sounds, false starts, and stammered repeats while keeping their words and phrasing. It runs after recognition, so it can reformat words but can't recover a word that was misheard.
How do I add a dictation feature to my app using an API?
Capture a bounded clip with a push-to-talk hotkey, send it in one HTTP request, and insert the returned text at the cursor. The Dictation API returns both the verbatim transcript and cleaned-up text in that single call. Alternatively, call the Sync API and run your own cleanup pass through an LLM.
Should I use a synchronous, streaming, or async API for a dictation feature?
Use a synchronous API for push-to-talk dictation, because bursts are short and bounded, and one HTTP request avoids WebSocket setup on every burst. Use streaming for long, continuous dictation where users want to watch words appear. Async (pre-recorded) APIs are too slow for interactive dictation.
What's the lowest-latency way to get from a keypress to finished dictated text?
Pre-warm the connection when the key goes down, send the clip to a sync endpoint, and use a small, fast model for cleanup. On AssemblyAI that's about 134 ms p50 for transcription, around 100 ms of warmed transport, and roughly half a second of cleanup. The total is under one second for a short clip.
Can a dictation feature handle medical terminology and PHI?
Yes, if you fix vocabulary at the recognition layer and put a BAA in place before processing PHI. Pass a prompt describing the clinical setting plus keyterms for drug names. AssemblyAI signs a Business Associate Addendum (BAA) for customers processing PHI; confirm which endpoints your BAA covers before sending patient audio.
