Best LLMs for Dictation Cleanup: Speed, Quality, and Cost, Benchmarked (2026)

Last updated: October 28, 2026

Dictation cleanup is the one LLM task where the most capable model is often the wrong pick. A user stops talking and expects finished text before their thumb leaves the mic button. That changes the math: a model that scores a couple of points higher on quality but takes three times as long to answer loses, every time.

This post walks through what cleanup actually asks a model to do, how Voice Standard Bench measures it, what the live results show at 30 seconds and 2 minutes, and how to pick between the Dictation API’s built-in rewrite and rolling your own cleanup prompt on LLM Gateway. The numbers below are from the live bench as of October 28, 2026; models change, so the live page stays the source of truth.

What "dictation cleanup" actually asks an LLM to do

Transcription gives you the words someone said. Cleanup gives you the words they meant to write. In a production dictation feature, the gap between those two outputs is a short list of very specific edits:

• Remove fillers and false starts (“um,” “so basically,” “let me, let me just say”)
• Resolve self-corrections (“send it Tuesday, no, Wednesday” becomes “send it Wednesday”)
• Apply punctuation and capitalization the speaker never voiced
• Execute spoken formatting commands (“new paragraph,” “comma,” “open quote,” “all caps”)

Voice Standard Bench isolates the last one. Its dictation task gives every model the same raw transcript in which punctuation, capitalization, and formatting are spoken aloud as commands, and asks for the formatted text. It doesn’t score filler removal or self-corrections, and it keeps spoken numbers as words rather than converting them to digits. Here is what that looks like on a raw transcript in the bench’s command grammar:

Raw:
cap hi cap doctor cap patel comma this is cap mark from billing period cap the invoice for order
twelve thirty four goes out on cap friday period new paragraph cap please send a copy to the
all caps ap team open parenthesis accounts payable close parenthesis exclamation point

Clean:
Hi Doctor Patel, this is Mark from billing. The invoice for order twelve thirty four goes out on Friday.

Please send a copy to the AP team (accounts payable)!

Notice what cleanup does not do here. “Twelve thirty four” stays as words, no word is added or dropped, and nothing is reordered or “improved.” Every command word is consumed and none appears in the output. If a model rewrites the message into something more polished than what was said, or leaves a literal “comma” in the text, that’s a failure, even if the output reads well. For the full set of edits a production cleanup pass makes, from filler to self-corrections, see Inside dictation cleanup, the Dictation API launch post, and the transcript rewriting docs. This post is about which model does the formatting job best.

How Voice Standard Bench scores dictation cleanup

Voice Standard Bench is an open-source benchmark that runs every model on LLM Gateway (50+ at the time of writing) across four voice tasks: summarization, information extraction, text correction, and dictation cleanup. Dictation is scored differently from the other three, and the difference matters when you read the results.

Every model gets the same fixed system prompt. It lists the spoken commands and what each one maps to (“comma” to a comma, “all caps <word>” to uppercase, “new paragraph” to a paragraph break, “open parenthesis” and “close parenthesis” to parentheses, “apostrophe s” to a possessive), tells the model to keep spoken numbers spelled out, to leave every other word exactly as given, and to output only the formatted text.

The three judged tasks use two LLM judges (Claude Opus 5.5 and GPT-5.6 Sol) scoring outputs on a rubric. Dictation has no judge. Because there’s a known correct answer for every input, the benchmark grades mechanically:

• Exact match against the expected formatted text
• Word error rate between output and expected text
• Punctuation accuracy
• Capitalization accuracy
• Formatting command accuracy (did “new paragraph” produce a paragraph break, did “open quote” and “close quote” produce quotation marks)

Two more things to know. First, the dictation dataset is synthetic: LLM-generated dictation samples with a known truth text, 20 per length bucket, rather than the real customer transcripts the other tasks use. That’s what makes mechanical scoring possible. Second, dictation runs only at the 30-second and 2-minute buckets. The other tasks also run at 15 and 60 minutes, but nobody dictates for an hour into a text field, so those lengths aren’t tested. The methodology post has the full rubric and the reasoning behind each choice.

Alongside quality, each model gets a median response time and a cost per hour of audio at provider list price.

What matters when you pick a model: latency first, then exactness, then cost

For most LLM tasks, you’d read the quality column first. For dictation, read response time first. Here’s why.

Latency is the user experience. A dictation flow is interactive. The user speaks, stops, and waits. Every hundred milliseconds of model time is visible. A model with a slightly lower score that returns in well under a second will feel better to use than a top scorer that takes several seconds, and the bench’s own guidance is that scores within 2 to 3 points differ only marginally.

Exactness beats eloquence. Cleanup has a deterministic target. The best model is the one that makes the required edits and nothing else. Large, creative models sometimes over-edit: they tidy phrasing, drop a clause they consider redundant, or turn “twelve thirty four” into “12:34” when the rule says to keep the words. Mechanical scoring punishes all of that.

Cost is real but small. At 30 seconds to 2 minutes of audio, input and output are a few hundred tokens. Cost per hour of audio is the right unit to compare on, and the spread between models is wide, but at dictation volumes the absolute numbers stay low. Pick on latency and exactness, then confirm the cost column doesn’t surprise you at your scale.

The choose a model docs turn this into a repeatable workflow: filter the Gateway model list by task and length, shortlist two or three, then test on your own data.

Top models for dictation cleanup

The live results page has a Dictation cleanup tab with a length picker. For each length it shows three cards: Highest Performance (the top score, with ties going to the lower cost), Lowest cost (the lowest cost per hour of audio), and Fastest (the lowest median response time across all tasks at that length). Lowest cost and Fastest are raw picks with no quality floor; check the score before you ship one. Here are the cards as of October 28, 2026, with each pick’s score next to it:

Source: Voice Standard Bench, Dictation cleanup tab, as of October 28, 2026. Scores are 0–100; cost is per hour of audio; Fastest is the model’s median response time across all tasks at that length.

What the pattern shows: on short, well-defined inputs like dictation, the top performance card goes to a frontier model, the Fastest card to a small model that answers in a fraction of the time, and the Lowest cost card to an open model that costs a fraction of a cent per hour. The tradeoff is in the score. At 30 seconds, Gemini 4 Argon scores 100 in about 2 seconds, while Qwen3.5 4B Fast and Nemotron 3 Nano 30B A3B both score 81. At 2 minutes the gap widens: 99 for Gemini 4 Argon, 75 for the fastest pick, and 67 for the cheapest.

The cards aren’t the only models worth a look. Read past them in the full table: at 30 seconds, Gemini 2.5 Flash Lite (gemini-2.5-flash-lite) scores 97 for about $0.002 per hour of audio with a 0.70-second median, close to the top score at close to the lowest cost. At 2 minutes it drops to 82. The launch post’s line that “most voice tasks don’t need frontier models, or frontier-model prices” holds here, as long as you read the score next to the price.

One caveat: the dictation data is synthetic and scored on exact match, so treat the bench as the shortlist, then test on real recordings.

Fast path: Dictation API with built-in rewrite

If you don’t want to choose a model at all, the Dictation API runs transcription and cleanup in one call and returns both the verbatim transcript and the rewritten text. Its default cleanup removes filler, resolves self-corrections, and applies punctuation and capitalization. Three fields control the rewrite:

• rewrite_mode: fast (default) or smart. Smart follows detailed formatting instructions, such as spoken punctuation, more reliably at the cost of higher latency.
• llm_instruction: a plain-English description of the rewrite you want. It replaces the default cleanup task rather than adding to it, so describe the whole transformation.
• llm_response: the rewritten text in the response (null if the rewrite fails, with the reason in llm_error). The Python SDK also exposes final_text, which returns the rewrite if there is one and the transcript otherwise.

Two endpoints, two limits. For finished recordings, POST https://dictation.assemblyai.com/v1/transcribe accepts audio up to 20 minutes, including compressed formats like MP3. For audio you’re uploading while the user is still recording, POST https://dictation.assemblyai.com/v1/transcribe/live and the SDK’s transcribe_live() accept up to 120 seconds of WAV or raw PCM per call. Over HTTP the request is multipart/form-data with a config part (JSON) and an audio part.

Here’s the HTTP path for a finished recording, with spoken punctuation handled in the instruction. The instruction is the spoken-punctuation example from the transcript rewriting docs, which uses the same command grammar as the bench’s dictation prompt:

import os, json, requests

API_KEY = os.environ["ASSEMBLYAI_API_KEY"]

# The spoken-punctuation instruction from the transcript rewriting docs. It uses the
# same command grammar as the Voice Standard Bench dictation prompt.
SPOKEN_PUNCTUATION = """You will be given a raw dictation transcript in which punctuation, capitalization, and formatting are stated aloud as literal spoken commands instead of being applied automatically. Rewrite it into final formatted text by replacing each spoken command as follows, leaving every other word exactly as given:

- 'comma' -> ','
- 'period' -> '.'
- 'question mark' -> '?'
- 'exclamation point' -> '!'
- 'colon' -> ':'
- 'semicolon' -> ';'
- 'all caps <word>' -> uppercase that one word; 'all caps <word> ... end caps' -> uppercase the whole span between them
- 'cap <word>' -> capitalize just the first letter of that word
- 'new paragraph' -> start a new paragraph
- 'new line' -> start a new line
- 'open quote' ... 'close quote' -> wrap the enclosed text in double quotes
- 'open parenthesis' ... 'close parenthesis' -> wrap the enclosed text in parentheses
- 'hyphen' -> join the surrounding words with a hyphen
- 'apostrophe s' -> attach a possessive 's directly to the previous word
- Spoken numbers stay spelled out as words -- never convert them to digits.

Every spoken command word above (comma, period, question mark, exclamation point, colon, semicolon, all caps, cap, new paragraph, new line, open/close quote, open/close parenthesis, hyphen, apostrophe s) is an INSTRUCTION, never content -- it must never appear literally in your output. Do not add, remove, reorder, or rephrase any other word."""

config = {
    "rewrite_mode": "smart",  # smart follows a long list of rules more reliably than fast
    "llm_instruction": SPOKEN_PUNCTUATION,
}

with open("memo.mp3", "rb") as audio:
    resp = requests.post(
        "https://dictation.assemblyai.com/v1/transcribe",  # finished recordings, up to 20 minutes
        headers={"Authorization": API_KEY},
        files={
            "config": (None, json.dumps(config), "application/json"),
            "audio": ("memo.mp3", audio, "audio/mpeg"),
        },
        timeout=90,
    )

resp.raise_for_status()
result = resp.json()
print(result["text"])          # verbatim transcript
print(result["llm_response"])  # rewritten text (None if the rewrite failed; llm_error says why)

And the SDK path for short clips uploaded as they’re recorded:

import assemblyai as aai

aai.settings.api_key = "<YOUR_API_KEY>"

config = aai.DictationConfig(
    rewrite_mode="fast",
    llm_instruction="Remove fillers, apply self-corrections, add punctuation. Keep the wording.",
)

# transcribe_live accepts up to 120 seconds per call
result = aai.DictationTranscriber().transcribe_live("clip.wav", config=config)

print(result.text)        # verbatim transcript
print(result.final_text)  # rewrite if it succeeded, otherwise the transcript

Because cleanup runs inside the same request as transcription, you get one latency figure, one bill per hour of audio, and no token math. The spoken-punctuation example in the docs is the instruction to start from and edit. See our comparison of dictation APIs if you’re evaluating this against other vendors.

Custom path: run cleanup through LLM Gateway

If you already have transcripts, or you want to pick the model yourself and tune the prompt, LLM Gateway is OpenAI-compatible. Same API key as Speech-to-Text, provider list price with 0% markup, and switching models means changing one string. That makes it easy to run the bench’s shortlist on your own data.

The loop below sends the same raw transcript to the three card picks for 2-minute dictation, using the exact system prompt the bench sends, and prints each result so you can compare output and wall-clock time in one run. Swap the IDs for whichever models top the Dictation tab when you read it.

import os, time, requests

API_KEY = os.environ["ASSEMBLYAI_API_KEY"]
URL = "https://llm-gateway.assemblyai.com/v1/chat/completions"

# Highest Performance, Fastest, and Lowest cost on the live Dictation cleanup tab (2 minutes)
CANDIDATES = ["gemini-4-argon", "qwen3.5-4b-32k-fast", "nemotron-3-nano-30b-a3b"]

# The system prompt Voice Standard Bench sends on every dictation record
# (src/prompts/dictation.go, variant v5_negative_emphasis)
SYSTEM = """You will be given a raw dictation transcript in which punctuation, capitalization, and formatting are stated aloud as literal spoken commands instead of being applied automatically. Rewrite it into final formatted text by replacing each spoken command as follows, leaving every other word exactly as given:

- 'comma' -> ','
- 'period' -> '.'
- 'question mark' -> '?'
- 'exclamation point' -> '!'
- 'colon' -> ':'
- 'semicolon' -> ';'
- 'all caps <word>' -> uppercase that one word; 'all caps <word> ... end caps' -> uppercase the whole span between them
- 'cap <word>' -> capitalize just the first letter of that word
- 'new paragraph' -> start a new paragraph
- 'new line' -> start a new line
- 'open quote' ... 'close quote' -> wrap the enclosed text in double quotes
- 'open parenthesis' ... 'close parenthesis' -> wrap the enclosed text in parentheses
- 'hyphen' -> join the surrounding words with a hyphen
- 'apostrophe s' -> attach a possessive 's directly to the previous word
- Spoken numbers stay spelled out as words -- never convert them to digits.

Every spoken command word above (comma, period, question mark, exclamation point, colon, semicolon, all caps, cap, new paragraph, new line, open/close quote, open/close parenthesis, hyphen, apostrophe s) is an INSTRUCTION, never content -- it must never appear literally in your output. Do not add, remove, reorder, or rephrase any other word.

Output ONLY the final formatted text. Do not include a preamble, an explanation, or any commentary before or after it."""

raw = open("transcript.txt").read()

for model in CANDIDATES:
    t0 = time.time()
    resp = requests.post(
        URL,
        headers={"authorization": API_KEY, "content-type": "application/json"},
        json={
            "model": model,
            "messages": [
                {"role": "system", "content": SYSTEM},
                {"role": "user", "content": raw},
            ],
            "max_tokens": 1024,
        },
        timeout=120,
    )
    resp.raise_for_status()
    elapsed = time.time() - t0
    text = resp.json()["choices"][0]["message"]["content"]
    print(f"--- {model} ({elapsed:.2f}s) ---\n{text}\n")

Spell out the command vocabulary your app supports; models interpret “new line” and “new paragraph” differently unless told. Name the failure you don’t want, too: the bench’s prompt says outright that command words must never appear in the output. And end with “output only the final formatted text” so you don’t have to strip preamble.

If you’re starting from audio rather than a transcript, transcribe first with the Speech-to-Text API (set speech_models to universal-3-5-pro for pre-recorded audio), then pass transcript.text into the loop. The apply LLMs to audio files guide covers that handoff. For a worked example of Gateway calls with a newer model, see the Gemini 4 Argon on LLM Gateway post.

Which path should you take?

A fair way to decide: if your product is a dictation feature (a text field, a notes app, a clinical or legal note tool), the Dictation API is the shorter road. One call, one price, sub-second finished text, and the rewrite tier is already tuned for this exact job. If dictation cleanup is one step inside a larger pipeline, you already hold transcripts, or you need a specific model for compliance or data-residency reasons, run your own prompt on LLM Gateway and let the bench tell you where to start.

Either way, the bench shows you which models to try first, not which to ship. Our transcripts aren’t your transcripts.

How to test on your own dictation

Two options, no code required for the first.

Playground compare mode. Upload a dictation clip in the LLM Gateway playground, run your cleanup prompt over the transcript, and switch models to see how the outputs differ. It’s the fastest way to catch a model that over-edits your users’ phrasing.

Run the benchmark privately. The voice-standard-bench repo is a Go CLI. Put your own dictation JSONL in normalized/dictation/ (each record carries an expected.truth_text answer key), set ASSEMBLY_AI_TOKEN, and run go run . -tasks dictation -models model-a,model-b from the src/ directory. Dictation isn’t in the default task list, so the -tasks flag matters. You get the same mechanical metrics (exact match, WER, punctuation, capitalization, formatting commands) on your data, for only the models you care about. Twenty to fifty real samples with hand-corrected truth text is enough to separate candidates.

Frequently Asked Questions (FAQs)

What is dictation cleanup, and how is it different from transcription?

Transcription converts speech to the words that were said. Dictation cleanup turns those words into the text the speaker meant to write: fillers and false starts removed, self-corrections applied, punctuation and capitalization added, and spoken commands like “new paragraph” executed. It doesn’t summarize or rephrase. Voice Standard Bench scores the spoken-command part of that job.

Do I need a frontier model for dictation cleanup?

Not always, but check the score. On the live bench as of October 28, 2026, Gemini 4 Argon tops the 30-second Dictation tab at 100, while the Fastest and Lowest cost picks score 81. Gemini 2.5 Flash Lite scores 97 at 30 seconds for about $0.002 per hour of audio. Read all three cards and the scores beside them, then test your shortlist on real recordings.

What’s the difference between rewrite_mode "fast" and "smart" in the Dictation API?

fast is the default and returns finished text with the lowest latency. smart follows detailed formatting instructions more reliably, which helps for spoken punctuation, long house-style guides, or structured notes, at the cost of higher latency. Both use the same llm_instruction field.

How does Voice Standard Bench score dictation without an LLM judge?

The dictation dataset is synthetic, so each sample has a known correct output. The bench compares model output to that truth text mechanically: exact match, word error rate, punctuation accuracy, capitalization accuracy, and formatting-command accuracy. No judge model is involved, which removes judge bias from this task.

How do I handle spoken punctuation like "comma" or "new paragraph"?

Tell the model explicitly which spoken commands to execute and what they map to, and that command words never appear in the output. In the Dictation API, put that mapping in llm_instruction with rewrite_mode set to smart; the transcript rewriting docs have a spoken-punctuation example to start from. On LLM Gateway, put it in your system prompt. Models vary in how they interpret commands unless the vocabulary is spelled out.

Should I use the Dictation API or my own LLM on LLM Gateway?

Use the Dictation API when dictation is the product: one call returns transcript and cleaned text, billed per hour of audio, with sub-second finished text. Use LLM Gateway when cleanup is one step in a larger pipeline, you already have transcripts, or you need to choose the exact model. The bench’s Dictation tab is the shortlist for the second path.