Automatic speech-to-text punctuation, casing, and ITN to boost transcript readability

A speech-to-text model that returns so we shipped universal three five pro on february fourth twenty twenty six and the error rate dropped to three point eight seven percent is technically correct and practically useless. Three things stand between that word stream and something you can put in front of a user: punctuation, casing, and inverse text normalization. All three are on by default on the AssemblyAI API, and all three run during transcription — they format what was said without changing it.

That is usually what you want. Sometimes it is not enough, because what was said is not what the speaker meant to send. This post covers both: the formatting parameters and how to control them, and the separate rewrite pass you reach for when the transcript needs to become clean prose rather than a faithful record.

What punctuation, casing, and ITN actually do

These three get bundled together as “formatting,” but they are distinct operations with different failure modes. Knowing which one produced a piece of output tells you which parameter to reach for when it is wrong.

Punctuation

Punctuation inserts sentence and clause boundaries: periods, commas, question marks, exclamation marks. The model predicts them from word sequence and phrasing rather than from silence alone, which is why a fast speaker who never pauses still gets sentence breaks and a slow speaker who pauses mid-clause does not get spurious ones.

Sentence boundaries matter more than they look. Downstream Speech Understanding features — sentiment analysis, topic detection, entity detection — operate on sentences. Strip the punctuation and you degrade every model that consumes the transcript, not just its readability.

Casing

Casing capitalizes sentence openers, proper nouns, and acronyms. This is the component most sensitive to vocabulary: the model has to know that “assembly ai” is a company and “AWS” is an initialism before it can case them. When casing is wrong on a domain term, the fix is almost always vocabulary bias rather than a formatting setting — see the keyterms_prompt section below.

Inverse text normalization (ITN)

ITN converts spoken forms into written forms. It is the step that turns february fourth twenty twenty six into February 4, 2026, four fifteen p m into 4:15 PM, and twenty five dollars into $25. The same pass handles phone numbers, currency, email addresses, and URLs.

ITN is the component you are most likely to want to disable, and for a specific reason: if a downstream system does its own number parsing, ITN’s output can fight it. A workflow that regex-matches spoken digit strings will break on $25. That is a legitimate reason to turn formatting off, and it is a one-line change.

How formatting behaves on Universal-3.5 Pro and Universal-3.5 Pro Realtime

Formatting behavior differs between the asynchronous and streaming paths, and the difference is structural rather than a setting someone forgot to expose.

Universal-3.5 Pro is the async flagship at $0.21/hr, covering 18 languages with native code-switching. On the async path the whole file is available before formatting runs, so the model has full future context when it decides where a sentence ends. Punctuation, casing, and ITN are all on by default and all can be turned off.

Universal-3.5 Pro Realtime is the streaming flagship at $0.45/hr base, same 18 languages, connecting over wss://streaming.assemblyai.com/v3/ws. On streaming, formatting is always on — there is no equivalent of punctuate: false. A live turn has to be emitted before the sentence is finished, so the model works with less future context than the async path has, and you will occasionally see a punctuation mark placed on a partial turn get revised as the turn completes. Plan for turns to be formatted, and do any downstream normalization yourself.

On accuracy: Universal-3.5 Pro posts a mean 3.87% normalized word error rate on short-form audio, first of nine systems, with the runner-up at 4.69%. Formatting quality rides on recognition quality — a model that mishears the word will not case it correctly either. The full numbers are on the benchmarks page.

Controlling formatting with the API

Two parameters, both defaulting to true on the async API.

punctuate controls punctuation and casing. format_text controls ITN and the rest of the written-form conversion. Note the dependency: format_text requires punctuate. Setting punctuate: false while leaving format_text: true is not a supported combination.

Default behavior — you do not have to pass anything:

{
  "audio_url": "https://example.com/quarterly-review.wav",
  "speech_models": ["universal-3-5-pro"]
}

Turning both off, for a raw word stream:

{
  "audio_url": "https://example.com/quarterly-review.wav",
  "speech_models": ["universal-3-5-pro"],
  "punctuate": false,
  "format_text": false
}

Note that async uses speech_models, plural, and takes a list. Streaming uses speech_model, singular. This trips people up when they port a config between the two paths.

The same request over the pre-recorded endpoint with cURL:

curl https://api.assemblyai.com/v2/transcript \
  --header "Authorization: $ASSEMBLYAI_API_KEY" \
  --header "Content-Type: application/json" \
  --data '{
    "audio_url": "https://example.com/quarterly-review.wav",
    "speech_models": ["universal-3-5-pro"],
    "punctuate": false,
    "format_text": false
  }'

Fixing casing on domain vocabulary with keyterms_prompt

When a product name, drug name, or surname comes back miscased or misspelled, the problem is recognition, not formatting. keyterms_prompt takes a list of exact terms and biases the model toward them, which fixes the spelling and the casing in one move.

{
  "audio_url": "https://example.com/standup.wav",
  "speech_models": ["universal-3-5-pro"],
  "keyterms_prompt": [
    "AssemblyAI",
    "Universal-3.5 Pro",
    "Kubernetes",
    "Sivakumar"
  ]
}

On the async path, keyterms are a $0.05/hr add-on with Universal-3.5 Pro and included with Universal-2. On Universal-3.5 Pro Realtime keyterms are included in the base rate. Full breakdown is on the pricing page.

For streaming, the config lives in the session parameters rather than a JSON body:

from assemblyai.streaming.v3 import (
    StreamingClient,
    StreamingClientOptions,
    StreamingParameters,
)

# Streaming: speech_model is singular. Formatting is always on.
client = StreamingClient(
    StreamingClientOptions(
        api_key="<YOUR_API_KEY>",
        api_host="streaming.assemblyai.com",
    )
)

client.connect(
    StreamingParameters(
        speech_model="universal-3-5-pro",
        keyterms_prompt=["AssemblyAI", "Universal-3.5 Pro", "Kubernetes"],
    )
)

Where automatic formatting gets it wrong

Three conditions account for most of the formatting errors people report, and each has a different response.

Degraded audio. Background noise and low-bandwidth channels hurt recognition first and formatting second. On streaming, voice_focus is the lever: near-field for headsets and phone audio, far-field for rooms, kiosks, and drive-thrus. Note the hyphens. It is a $0.10/hr add-on.

Overlapping speakers. When two people talk over each other, sentence boundaries get ambiguous because the model is reconstructing two interleaved grammatical structures. Diarization helps you attribute the words correctly afterward — averaged across DiPCo, CALLHOME, NOTSOFAR and AMI, Universal-3.5 Pro posts cpWER 30.17, with Azure closest at 30.35, then ElevenLabs Scribe V2 at 35.26, Speechmatics at 36.60, Gladia at 36.88 and Deepgram at 37.93 — but it will not un-blur a boundary in the overlap itself.

Domain jargon. Unfamiliar vocabulary produces wrong words, and wrong words get wrong casing. This is the keyterms_prompt case above, and it is the highest-leverage fix of the three.

There is also a fourth condition that is not really an error at all, and it is the rest of this post: the formatting is correct, and the transcript still is not what you want to send.

When formatting isn’t enough: rewriting the transcript

Punctuation, casing, and ITN are formatting. They put marks and capitals and written-form numbers around the words that were actually spoken. They do not remove a word, and they do not resolve a contradiction.

Which means a perfectly formatted transcript of real speech still looks like this:

Um, so can we, uh, move the, the meeting to Thursday? I think Friday works better, actually.

Every mark in that sentence is in the right place. It is still not a message anyone would send. Speech contains filler, restarts, and self-corrections, and the speaker’s intent is the last thing they said, not the average of everything they said. Transcription models are verbatim by design — but what was said is not what anyone wants to send.

The Dictation API adds a second pass for exactly this. It is speech-to-text plus an LLM cleanup step in a single call, and it returns:

Can we move the meeting to Friday? That works better.

Filler gone. The duplicated “the, the” gone. The self-correction resolved to Friday, which is what the speaker landed on. Tone preserved.

What comes back

The important design decision is that the rewrite does not replace the transcript. A response carries both:

• text — the verbatim transcript. Always present, never altered by the LLM.
• llm_response — the rewritten text, or null if the rewrite did not complete.
• llm_error — “timeout”, “error”, or null.

The Python SDK also exposes final_text, which is the cleaned-up text falling back to the verbatim transcript. That is the field most applications want, because it means the degraded path is still a usable path.

Shaping the rewrite with llm_instruction

Cleanup runs by default — omit llm_instruction and you get the default cleanup task. Pass one and you replace it with your own plain-English rewrite instruction, up to 2,048 characters. Here is the canonical clinical example:

import assemblyai as aai

aai.settings.api_key = "<YOUR_API_KEY>"

config = aai.DictationConfig(
    stt_prompt="A doctor dictating a patient visit note.",
    keyterms_prompt=["amoxicillin", "lisinopril", "metoprolol"],
    llm_instruction=(
        "Remove filler words and rewrite as a concise clinical chart note."
    ),
)

result = aai.DictationTranscriber().transcribe_live("clip.wav", config)

transcript = result.text
rewrite = result.final_text  # the cleaned-up text, falling back to the transcript

Three parameters are doing three different jobs here, and it is worth separating them. stt_prompt describes the situation so the model knows what kind of speech to expect, up to 6,000 characters. keyterms_prompt supplies exact strings to bias toward — drug names, in this case — up to 100 terms. llm_instruction describes the rewrite task. More detail in the prompting and keyterms docs and the transcript rewriting docs.

One safety property worth knowing: dictated commands are never executed. The transcript is passed to the model as fenced data, so a user who says “translate this into French” gets that sentence rewritten as speech, not carried out as an instruction.

Handling the rewrite deadline

The rewrite is best-effort with a 5-second internal deadline. If it does not land in time, the request still returns 200, with llm_response: null, llm_error: “timeout”, and the transcription intact.

This is the part people get wrong in integration: a non-null llm_error is not a failed request. You got a transcript. Handle it like this:

result = aai.DictationTranscriber().transcribe_live("clip.wav", config)

if result.llm_error:
    # 200 response, rewrite unavailable, transcript still valid.
    log.warning("rewrite unavailable: %s", result.llm_error)
    output = result.text          # verbatim fallback
else:
    output = result.llm_response  # cleaned-up text

insert_into_editor(output)

Set your client timeout to 90 seconds rather than something tight, so a slow rewrite does not get killed client-side before the server’s own fallback fires. The error handling docs cover the rest of the failure surface.

What it costs and what it fits

The Dictation API is $0.62/hr flat, and per the pricing page: “Every feature is included in the $0.62/hr rate. One line on your bill covers the whole request — there is no second rate to model and no token math to do.” Volume discounts apply at any tier. It runs on Universal-3.5 Pro, accepts 32 language codes, and returns finished text in about 0.36 s — with request_time_ms on every response so you can measure it yourself from your first request onward.

The constraint to design around is the 120-second cap per call, with WAV or raw 16-bit PCM only — compressed formats are rejected with a 415. That covers a message, an order, a short chart note. For anything longer, route to pre-recorded transcription and do the rewrite as a separate step.

Spoken punctuation: when your users say “comma” and “period”

There is a class of user for whom automatic punctuation is not a feature but an obstacle, and it is worth being direct about it.

Clinicians are trained to dictate punctuation out loud. A radiologist or a physician working from a traditional dictation system says:

“Medications colon aspirin comma naproxen period.”

…and expects:

Medications: aspirin, naproxen.

This is a genuine tension with automatic punctuation. AssemblyAI’s models apply punctuation automatically, and they handle spoken command words inconsistently — sometimes transcribing “comma” as the literal word, sometimes converting it. There is no verbatim or spoken-form mode on the AssemblyAI API today. It is the single most-requested capability in medical dictation and it is a real gap in the product.

It is also worth saying plainly that a competitor ships it: Deepgram has a documented dictation=true flag that converts spoken punctuation commands into the corresponding marks, and they market it in their medical content. If spoken punctuation is a hard requirement for your users, that is a real difference and you should weigh it.

The documented workaround

What you can do today is run an LLM pass over each finalized turn to interpret and strip the command words. This is the pattern documented in the medical scribe best practices guide, and it works through the LLM Gateway:

import requests

GATEWAY = "https://llm-gateway.assemblyai.com/v1/chat/completions"
MODEL = "qwen3.5-4b-32k-fast"  # Qwen3.5 4B Fast — AssemblyAI-hosted, 32,768-token context

SYSTEM = (
    "You convert dictated punctuation commands into punctuation marks. "
    "Words like 'comma', 'period', 'colon', 'new paragraph' and "
    "'new line' are commands, not content: replace each with the mark it "
    "names. Do not add punctuation the speaker did not dictate. Do not "
    "reword, summarize, or correct anything else. Return only the text."
)

def apply_spoken_punctuation(turn_text: str, api_key: str) -> str:
    response = requests.post(
        GATEWAY,
        headers={"Authorization": api_key},
        json={
            "model": MODEL,
            "temperature": 0,
            "messages": [
                {"role": "system", "content": SYSTEM},
                {"role": "user", "content": f"```\n{turn_text}\n```"},
            ],
        },
        timeout=30,
    )
    response.raise_for_status()
    return response.json()["choices"][0]["message"]["content"]

Two honest caveats. First, this is a workaround, not a native mode — you are paying an extra round trip and inheriting an LLM’s judgment about which “period” is a command and which is a word in “grace period.” Second, it runs after the model has already applied its own punctuation, so your prompt has to tolerate marks that are already there.

The model here is worth a note. Qwen3.5 4B Fast (qwen3.5-4b-32k-fast) is the only AssemblyAI-hosted model on the LLM Gateway roster — it runs on AssemblyAI’s own GPUs, which is exactly why it is fast and cheap for cleanup work: 612 ms average on voice rewrite tasks, 1.9× faster than GPT-4.1, 94% cheaper per hour of audio, at $0.10 / $0.50 per million prompt / completion tokens with a 32,768-token context. It is a rewrite model rather than an agentic one, so it supports max_tokens, temperature, and stream only — there is no tools, tool_choice, or response_format. For stripping punctuation commands out of a finalized turn, that is the whole parameter surface you need.

Run it per finalized turn rather than per session, keep the temperature at zero, and test it against real dictation from your own users before shipping.

Choosing between formatting, rewriting, and raw output

The short version: formatting is what the model does to the words. Rewriting is what a second model does to the sentence. Most applications want the first; anything where the output goes straight into what the user is writing wants the second.

Frequently asked questions

What’s the difference between punctuation, casing, and ITN?

Punctuation adds grammatical marks like commas, periods, and question marks. Casing handles capitalization for sentence openers, proper nouns, and acronyms. Inverse text normalization (ITN) converts spoken-form numbers, dates, currency, and contact details into their standard written form — “february fourth twenty twenty six” becomes “February 4, 2026.” On the AssemblyAI API, punctuate controls the first two and format_text controls the third.

How accurate is automatic punctuation compared to manual editing?

Formatting accuracy tracks recognition accuracy, because a misheard word cannot be punctuated or cased correctly. Universal-3.5 Pro posts a mean 3.87% normalized word error rate on short-form audio, first of nine systems. The remaining formatting errors cluster in three places: noisy audio, overlapping speakers, and unfamiliar domain vocabulary — the last of which you can fix yourself with keyterms_prompt.

Can I dictate punctuation out loud, like “comma” and “period”?

Not natively. AssemblyAI’s models apply punctuation automatically and handle spoken command words inconsistently, and there is no verbatim or spoken-form mode on the API today. Deepgram does ship a documented dictation=true flag that converts spoken commands into punctuation. The workaround on AssemblyAI is to run each finalized turn through the LLM Gateway with a prompt that replaces command words with their marks, as documented in the medical scribe best practices guide.

Can I customize automatic punctuation rules for my specific use case?

You cannot edit the punctuation rules directly. You have two levers instead: keyterms_prompt, which biases the model toward exact domain terms and fixes their spelling and casing, and a post-processing pass through the LLM Gateway for anything rule-shaped. If what you actually want is different prose rather than different marks, the Dictation API’s llm_instruction lets you specify the rewrite in plain English.

Does automatic punctuation work the same across different languages?

No. Punctuation and ITN follow each language’s own conventions — decimal separators, quotation marks, date order, and sentence-final marks all differ. Universal-3.5 Pro covers 18 languages with native code-switching on the async and streaming paths, the Dictation API accepts 32 language codes, and Universal-2 covers 99+ languages for legacy integrations.

What happens to punctuation accuracy with background noise or poor audio quality?

It degrades, because recognition degrades first. On streaming, voice_focus is the mitigation: set near-field for headsets and phone audio, far-field for rooms, kiosks, and drive-thrus — note the hyphens. It is a $0.10/hr add-on. For pre-recorded audio, the practical step is to test with recordings captured the same way your production audio will be, rather than with clean studio samples.