How to evaluate a dictation API in 2026

You've been asked to ship dictation. You have a shortlist of speech-to-text vendors, a word-error-rate leaderboard somebody sent you, and two weeks to pick one.

The shortlist was built for the wrong workload.

Generic speech-to-text comparisons are written around long recordings — meetings, sales calls, interviews. A vendor can win that leaderboard on hour-long audio and still be the wrong choice for an eleven-second utterance a user is staring at while the cursor blinks. Dictation is a different shape of problem: single speaker, short clip, start and stop controlled by a keypress. And the part the generic guides miss — what you're buying isn't a transcript. It's finished text, in one call, on one line of your invoice.

That changes what you should test. Word accuracy still matters enormously, but it stops being the whole question once output has to arrive shaped, punctuated, and free of the noise a human makes while thinking out loud.

So: five criteria that separate dictation APIs from transcription APIs, the test to run for each, how the market sorts, and when to choose something that isn't an API at all. The five follow from what a dictation feature needs to work inside a real product.

Everything below is testable on your own audio today. Where we describe our own endpoint, the Dictation API is live and every claim points at the docs or the pricing page so you can check it rather than take it.

The five criteria, at a glance

Criterion 1 — Cleanup quality: does it return finished text?

Eighteen months ago this criterion didn't exist, because no API offered it. Today it's the line between a dictation API and a transcription API you'll have to finish yourself.

Score three things. Does the response remove filled pauses, resolve false starts, and collapse stammered repeats — while leaving everything else exactly as spoken? Can you steer the output per request, or is there one fixed behavior you take or leave? And does it ever rewrite meaning it had no business touching? A pass that quietly rephrases a dosage into something more fluent is worse than no pass at all, and a demo of five clips won't surface it.

Now the boundary that belongs in every serious evaluation of this criterion, because getting it backwards is how teams buy the wrong thing: cleanup polishes, it does not correct. Hand a rewrite pass a misheard drug name and it hands back a beautifully punctuated wrong drug name — it works with the words it was given, and it can't recover a term the speech model never heard. The mechanics of dictation cleanup go through that boundary in detail. A fluent wrong answer is worse than a messy right one, because it reads authoritative: a clinician skims it, a customer signs it, nobody notices. Which is exactly why the transcript underneath is criterion 3, and why you score the two together. Cleanup is not where accuracy comes from. The speech model is. So if a vendor's accuracy story is really a story about their rewrite layer, that's an answer to a different question.

The test is cheap. Collect twenty utterances that sound like your users on a bad day — thinking out loud, restarting sentences, correcting themselves mid-word. Record them once, send the same audio to every vendor, and diff each output against what you meant to say rather than against a reference transcript, which rewards verbatim. Your users don't.

Two things to establish about default behavior, because vendors differ and it changes your integration. First, is cleanup on when you send nothing? On the Dictation API it is: a request with an empty config applies the default pass — filler sounds, false starts, and stammered repeats removed, every other word, spelling, and mark left exactly as spoken.

Second, and this is the question most evaluations skip: can you turn it off? On the Dictation API, not really — setting llm_instruction to null means the same as omitting it, so the default cleanup still runs. What you get instead is both versions in every response, which answers the same need from the other direction: text is always the verbatim transcript and is never touched by the rewrite, so a product that needs the words as spoken reads text and ignores llm_response. Ask every vendor which of those two shapes they offer, because "you can disable it" and "you always get both" are different integrations.

Then test steerability, because that's where dictation stops being one feature and becomes a product surface. The same audio should come back as a chart note, as a booking confirmation, or as a JSON object with the fields your form needs — decided per request. On the Dictation API that control is llm_instruction: plain English, up to 2,048 characters, set per request, replacing the default cleanup task rather than adding to it.

If you'd rather feel this than read it, Blurt is our free, open-source macOS dictation app. Bring your own API key, hold a hotkey, and watch finished text land in whatever app you're in. It's one Dictation API call per utterance and the source is right there, which makes it the fastest way to run the twenty-utterance test against your own speech.

Criterion 2 — Latency, measured the way users experience it

Reframe this one before you test it. The number you care about isn't streaming latency or time to a partial result. It's time to finished text: the gap between the user releasing the hotkey and the finished sentence appearing where they're looking.

Dictation is a UI-blocking interaction. There's no background job to hide behind and no spinner you can get away with, because the user has stopped talking and is watching the cursor. Miss by two seconds and they go back to typing — the failure mode nobody puts in a dashboard.

Four things to measure.

Total round trips, not per-hop timings. A pipeline that transcribes with one vendor and rewrites with another pays two round trips, the second across the public internet. Vendors quote the hop they own; measure the wall clock your user experiences.

A distribution, not a hero number. Run a few hundred real clips and report p50 and p95; a best-of-twenty figure from a quiet afternoon tells you nothing about Monday morning. It's also where you find out whether a vendor's speed comes from routing you to a smaller model — ask directly, because a quicker model that misses more entities loses criterion 3 to win criterion 2. Our model underneath is Universal-3.5 Pro, the same flagship behind our pre-recorded and realtime endpoints, not a stripped-down variant.

Whether upload sits inside the wait. A clip-based API that only starts work once the whole file has arrived makes the user pay for the upload after they've stopped talking. The Dictation API reads the body as it arrives, so a client that streams audio during the recording leaves only the last stretch to process — worth measuring on your own connection, because the saving grows with clip length and is negligible on very short clips.

What happens on the slow path. Almost nobody asks, and it decides how your UI behaves in the tail. On the Dictation API the rewrite carries a 5-second internal deadline; exceed it and you still get HTTP 200 with the verbatim transcript and llm_error: "timeout". The worst case your user sees is an unpolished sentence rather than a wait. Ask every vendor what their equivalent is — "it retries" isn't something you can build a text field on.

For reference: typical short clips respond in under one second, and every response carries request_time_ms, sync_time_ms, and audio_duration_ms, so you can measure the split yourself. If your workload is transcription with no rewrite, the Sync API sits underneath at roughly 134 ms p50 on the same model.

Criterion 3 — Jargon accuracy, not word accuracy

Once you know the output shape and the timing, this is the criterion that predicts whether people keep using the feature. It's also the one aggregate word error rate hides most effectively.

Here's why. A dictated sentence is short, so one missed word is a large fraction of it — and the words people dictate are the ones a general model finds hardest: drug names, package names, version strings, internal codenames, customer surnames. A model can post an excellent aggregate score by getting every "the" and "and" right while mangling the four tokens that carried the meaning. Aggregate accuracy averages your product's vocabulary into the noise floor.

So test the metric that matters: Missed Entity Rate. We've covered entity accuracy in speech-to-text at length, and our own coverage of how speech-to-text accuracy is measured handles the methodology, so I won't rebuild either. For your evaluation: build twenty utterances stuffed with your own domain vocabulary and count entity misses rather than word misses.

Look too at what controls each vendor gives you to move that number before any rewrite layer runs. The Dictation API exposes two, and both steer transcription rather than the rewrite: stt_prompt, a description of what the model is about to hear, up to 6,000 characters; and keyterms_prompt, an explicit term list, up to 100 terms and 8,000 characters across all of them. Ask your shortlist what theirs are, and what each one costs you elsewhere — every vendor has tradeoffs there, and few are written down.

One integration detail worth knowing before you build against it: both fields accept legacy aliases — prompt for the first, keyterms or word_boost for the second — but you may send only one name per field, and the config rejects unknown fields outright with a 400 rather than ignoring them. That's the friendlier failure mode, and it's the opposite of what most APIs do, so a typo surfaces as an error in your logs instead of a setting that quietly does nothing. Check which behavior each vendor on your list has; silent no-ops are what turn a bad prompt into a week of debugging.

The framing that keeps these two surfaces straight is the one our docs use: stt_prompt tells the speech model what it's about to hear; llm_instruction tells the rewrite what to do with the transcript after it's written down. Putting a drug name in the instruction does nothing — it was never going to be heard in the first place. It goes in keyterms_prompt.

On numbers: Universal-3 Pro, the prior-generation flagship, records a 5.6% English mean word error rate across 250-plus hours of test audio. That's a model-level fact about pre-recorded audio, and I want to be precise about what it isn't — it is not a dictation benchmark, it wasn't measured on short single-speaker clips, and you shouldn't buy on it. Our benchmarks page carries the published figures with their methodology attached.

Criterion 4 — Integration effort

This criterion gets waved through in most evaluations, which is odd — it decides whether the feature ships this quarter or next.

On one side, a single POST with two form parts: config, a JSON object, and audio, the bytes. Content type multipart/form-data, audio as audio/wav or audio/pcm, the raw API key in the Authorization header with no Bearer prefix. There's a Python SDK that wraps the endpoint as DictationTranscriber; in every other language you call it over HTTP, which for most teams makes the integration a function rather than a dependency.

On the other side, the build path: wire a transcription vendor to an LLM vendor and own everything between them. That's a real option, but price it honestly. You're adding a second inference call, a second failure mode, a second rate limit, a second security review, and a cleanup prompt you'll maintain indefinitely — because that prompt is a product surface, and the version that works specifies rule by rule which words to remove and, harder, which to never remove. Then the engineering nobody scopes: connection pre-warming, fallback when the second hop times out, and deciding what your text field renders when the rewrite fails but the transcript succeeded.

The argument here is time, not tidiness. Not "one vendor instead of two" — that's a procurement conversation, and it isn't yours to have. The engineering question is how many weeks the glue costs, and who's paged when the second hop degrades. If that tradeoff is the real decision in front of you, bundle or build it yourself is the fuller argument. If the cleanup behavior is your differentiator, that work is your product. If it's plumbing, every week on it is a week off the feature users asked for.

Three facts to check against every vendor. The clip ceiling — 120 seconds per request here, an architectural fact rather than a gotcha: push-to-talk utterances sit well inside it, but a fifteen-minute clinical note needs chunking on natural pauses. The accepted formats — this endpoint decodes audio as it arrives, so compressed formats can't be decoded incrementally and are rejected with a 415; if your recorder emits WebM or M4A, budget for a decode step, which is the sort of thing that surfaces in week three rather than week one. And the error surface — a missing or invalid API key returns 401, and errors raised while parsing the request itself use a different response body than validation errors do, so your handler needs to read both shapes. Read every vendor's error table before you scope, because that table is what your integration is mostly made of.

Criterion 5 — Compliance and data flow

This belongs inside the evaluation rather than after it: if your product touches clinical, legal, or financial dictation, this criterion can eliminate a vendor you've already built against.

Four questions, in writing.

Where does the audio go? Specifically: does anything leave the vendor's own infrastructure? For anyone bundling a cleanup pass this is the question that matters most, because a rewrite routed to a third-party model provider means your audio-derived text reaches a company you never evaluated. On the Dictation API, post-processing runs on a model we host; no third-party LLM provider is in the path.

What's retained, and for how long? Ask for the default, whether it's configurable, and what "deleted" means operationally.

Which regions can you pin to? Data residency becomes a contract term fast. The Dictation API runs in US West and EU North.

Can you get a Business Associate Addendum? If you're processing protected health information, that's a gate rather than a nice-to-have. AssemblyAI is considered a business associate under HIPAA, and we offer a standard Business Associate Addendum (BAA) that is required under HIPAA to ensure that AssemblyAI appropriately safeguards PHI. Paid accounts can review and sign it self-serve from the Data Controls page, at no additional cost — see the BAA FAQ. In a vendor comparison, the row you want reads BAA Available — and if a vendor answers with a compliance badge instead of a document their legal team will sign, that's a different answer than it looks like.

Get all four answers before the pilot. Retention defaults and subprocessor lists are what most often send a finished integration back to the start.

Best dictation APIs in 2026, compared

Here's how the market sorts on dictation-specific axes rather than the generic price-and-accuracy grid every speech-to-text roundup already carries.

Two things it deliberately doesn't do. It carries no head-to-head accuracy or latency figures against a named vendor, because nobody publishes a dictation benchmark and a number we generated in-house isn't evidence you should spend money on. And where a vendor hasn't published a figure, the cell reads "Not published" rather than an estimate — treat any table that fills those cells in with suspicion, including ours.

Consumer apps aren't in it either. Superwhisper, Dragon, and the leading dictation app are finished products rather than APIs — a different buying decision, and I'll come back to it.

Gladia's Audio-to-LLM is the closest structural comparison to a dictation API — transcription and LLM prompts in one job is the same idea. The difference is delivery: the pre-recorded API hands you a job ID and you poll, take a webhook, or supply a callback URL, and the LLM step bills per token on top of the hourly base rate. For a user watching a cursor, "we'll call your webhook" is a different product.

Deepgram's dictation parameter deserves precision, because the shared word causes real confusion. It "converts spoken dictation commands into their corresponding punctuation marks" — English-language, and it needs punctuate=true. That's different from a pass that removes filled pauses and repairs false starts. If your requirement is physicians speaking their punctuation, that parameter is closer to what you want than any cleanup layer.

ElevenLabs ships the closest competitor flag to a cleanup pass. Scribe v2's no_verbatim has the model remove "filler words, false starts and disfluencies from the transcript" — a real filler-removal control rather than a steerable rewrite, and worth testing on your own messy clips instead of dismissing. They also shipped Scribe v2 Medical in September 2026, at the same rate as Scribe v2, which is worth a look if your dictation is clinical. Note the BAA is restricted to Enterprise tier, and their HIPAA page scopes coverage to ElevenLabs Agents without saying whether standalone Scribe is included.

Google is the one to re-check between now and your decision. Chirp 3's Custom prompt feature is in Preview and takes formatting instructions in natural language, which is the nearest thing to a steering surface among the hyperscalers. It shapes how the transcript is written rather than reshaping it afterwards, so it isn't a cleanup pass — but the gap between "no instruction surface at all" and "a preview one" is the gap this whole criteria list is about, and Preview features move.

And the price beat, last because it belongs last. Competitor rates were checked against each vendor's public pricing page shortly before publication, and several move — Deepgram's streaming figure is promotional, and ElevenLabs' realtime price differs between their pricing and landing pages — so confirm before you model anything, and see our speech-to-text API pricing breakdown for the generic grid. What's worth modelling is variance: a bundled rate, billed per second with post-processing on one line, is forecastable, while a base rate plus per-token LLM billing is what surprises finance in month four.

When to choose something else

The criteria above sort dictation APIs against each other. They don't tell you whether you want one, and for a meaningful share of readers the answer is no.

Your audio has multiple speakers, or runs long. Meetings, interviews, calls, recorded consultations — anything where you need to know who said what, or where the recording is measured in tens of minutes. That's pre-recorded transcription with diarization, not a short-clip endpoint. Same for PII redaction or speech understanding across a whole recording.

You need words appearing as the user speaks. Progressive, word-by-word display is a streaming problem — a clip-based API can't render text before the clip ends, by definition. If your interface shows partial results while someone is still talking, evaluate streaming transcription instead.

You need verbatim, and cleanup would destroy meaning. Some medical records need the words as spoken, because formatting can carry clinical meaning — especially when a downstream model reads the note. Physicians are trained to speak their punctuation, so "patient open parentheses Andrew close parentheses is tired" is a rendering instruction rather than a filler pattern. Legal and compliance transcription has the same requirement. You can serve this from a dictation API by reading text, which is always the untouched transcript — but you'll still be paying for and receiving a rewrite you throw away, so a pure transcription endpoint is usually the cleaner fit.

The cleanup behavior is your differentiator. If the way your product turns speech into text is why customers pick you, own it — the prompt, the model, the evaluation set. You'll move slower and you'll be right to. Related: if you need a specific model on the rewrite layer, we don't offer model choice there today. It's the request we hear most and it's on the roadmap; watch the changelog rather than taking a roadmap answer as a product answer.

You want a dictation app, not a dictation API. If nobody on your team is going to write the integration, buy the finished product. Superwhisper, Dragon, and the leading dictation app all solve dictation for an individual at a desk and require you to ship nothing. An API is the right answer when dictation has to live inside your product, follow your vocabulary, and behave differently in a chart note than in a chat box.

Where this criteria list goes next

The interesting thing about writing down five criteria is watching which one moves.

Cleanup quality will fragment. Right now the question is whether a vendor has a rewrite layer at all — a yes-or-no column. Within a year it'll be a question about control: which model runs it, whether you can bring your own, whether it can be pinned so a prompt change on the vendor's side never changes your product's output. Model choice there is the ask we hear most, and it isn't in the initial release — being straight about that is worth more than a clean sweep of the table above.

The criterion I'd watch hardest, though, isn't on the list yet: predictability. Not median latency, not tail latency, but whether the same utterance produces the same shape of output tomorrow. Dictation is a habit, and habits are built on consistency more than on speed. Teams evaluating in 2027 will ask for output-stability guarantees the way they ask for uptime today, and most of us aren't ready for that question.

Which is a good reason to run these tests on your own audio rather than trusting the table.

Frequently asked questions

What is the best AI for dictation?

There's no single answer, because dictation splits into two purchases. To dictate into your own documents, buy a finished desktop app. To build dictation into a product, you want an API that returns finished text rather than a raw transcript — evaluated on cleanup quality, time to finished text, jargon accuracy, integration effort, and data-flow terms, in that order.

Does AssemblyAI have a dictation API?

Yes. The AssemblyAI Dictation API is generally available and returns finished text in a single request — a verbatim transcript in text and a cleaned, steerable version in llm_response, from one POST. It runs on Universal-3.5 Pro, caps clips at 120 seconds, supports 32 languages, and is priced at $0.62 per hour all-in. There's a Python SDK; every other language calls it over HTTP. You can try it in the playground under the Build tab before writing any code.

How much does the AssemblyAI Dictation API cost?

$0.62 per hour of audio, all-in — transcription and post-processing on one line item, billed per second with no minimum clip charge. There's no separate per-token cost for the cleanup pass and no 10-second billing floor. Check the pricing page for the current rate card before you model against it.

What languages does the Dictation API support?

32: English, Spanish, German, French, Italian, Portuguese, Turkish, Dutch, Swedish, Norwegian, Norwegian Nynorsk, Danish, Finnish, Estonian, Catalan, Galician, Romanian, Russian, Persian, Hindi, Marathi, Urdu, Vietnamese, Arabic, Hebrew, Japanese, Korean, Mandarin, Cantonese, Afrikaans, Zulu, and Xhosa. Set them with language_codes, which takes a list and defaults to English — pass several codes when a single clip moves between languages, which is common in dictation from bilingual speakers.

What is the most accurate dictation app?

No vendor publishes a dictation-specific accuracy benchmark, so any ranking you find was measured on a different workload — usually meetings or call recordings. The figure that predicts satisfaction on short utterances is Missed Entity Rate rather than aggregate word error rate, because one mangled drug name breaks a dictated sentence in a way a missed article never does. Build a test set from your own vocabulary and measure it yourself.

Which is better, AssemblyAI or Deepgram?

It depends which problem you're solving, and the honest comparison is structural rather than a benchmark. The AssemblyAI Dictation API returns a verbatim transcript and a rewritten, steerable version in a single response, with the rewrite running on a model we host. Deepgram's dictation parameter converts spoken dictation commands into punctuation marks, English-language — a different capability that some workflows need more. Neither vendor publishes a dictation benchmark, so test both on your own clips.

How do I evaluate and choose a dictation API for my product?

Score five criteria in order: cleanup quality, time to finished text, jargon accuracy, integration effort, and compliance and data flow. Run one test per criterion on your own audio — twenty messy utterances diffed against what you meant, p50 and p95 across a few hundred real clips, twenty jargon-heavy utterances counted for entity misses, a time-boxed integration spike, and a written data-flow answer for your security reviewer.

Is it worth paying for built-in LLM formatting, or should I combine a transcription API with GPT myself?

Buy the bundle when the cleanup layer is plumbing, and build it when the cleanup layer is your product. Building means a second inference call across the public internet, a second rate limit and failure mode, a second security review, and a cleanup prompt you'll maintain indefinitely. The bundled path is one round trip and one line item at $0.62 per hour all-in.