One parameter, 20% fewer missed entities: a before/after tour of Medical Mode

One parameter. That's the entire change. You add domain: "medical-v1" to a request you're already making, and the transcript that comes back gets measurably better at the words a clinician would care about. Missed Entity Rate drops to 3.2% — the lowest across benchmarked providers.

Percentages are abstract, though. What people actually want to see is the transcript. So this post is a tour: the same clinical audio, transcribed without Medical Mode and with it, so you can look at the specific words that change and decide for yourself whether the delta matters for your product.

The short version of what you'll see below: general-purpose speech models don't fail randomly on clinical audio. They fail in a very consistent pattern — they substitute a rare medical term with a common English word that sounds similar, and they do it fluently enough that the sentence still reads fine. That's the dangerous failure mode, because nothing in the output signals that something went wrong.

Everything here applies to both flagship models: Universal-3.5 Pro for pre-recorded audio and Universal-3.5 Pro Realtime for streaming.

What changes in the transcript

The examples below are illustrative — constructed to show the classes of error Medical Mode addresses, not verbatim output from a specific benchmark run. If you want real numbers on real audio, the benchmarks page has the methodology and the playground will run your own file in about a minute.

Drug name substitution

This is the most common and most consequential class.

Without Medical Mode: "We'll keep her on the Lamisil for the seizures and recheck levels in six weeks."

With Medical Mode: "We'll keep her on the Lamictal for the seizures and recheck levels in six weeks."

Lamisil is an antifungal. Lamictal is an anticonvulsant. The sentence is grammatical either way, an LLM summarizing it will produce a confident plan section either way, and a reviewer skimming twenty notes will not catch it.

Dosage and unit handling

Without: "Start the metoprolol at twenty five milligrams be I D."

With: "Start the metoprolol at 25 mg b.i.d."

Less dramatic, more annoying. Anything downstream that parses dosages — an order set, a medication reconciliation step, a structured field in your note template — needs the units in a canonical form. Medical Mode handles the abbreviations and numeric conventions clinicians actually speak.

Procedure and anatomy terms

Without: "Prior history of a cholecystectomy and a left total knee in twenty nineteen."

With: "Prior history of a cholecystectomy and a left total knee arthroplasty in 2019."

General models tend to truncate procedure names to the part they recognize. The truncation reads naturally, so nobody flags it, and the specificity a coder needs is simply gone.

Abbreviations and acronyms

Without: "Rule out D V T, check the eye NR before we bridge."

With: "Rule out DVT, check the INR before we bridge."

Clinical speech is dense with letter strings. A general model spells them out phonetically or, worse, resolves them into English words — "eye NR" is exactly the kind of thing that turns a searchable term into noise.

Speaker attribution in the encounter

This one isn't Medical Mode specifically; it's the diarization in Universal-3.5 Pro, and it matters just as much for clinical audio. It's the most accurate diarization AssemblyAI has shipped, optimized for cpWER rather than DER — meaning it's scored on whether the right words land with the right speaker, not just on where segments break. In practice it holds up on the short turns and overlapping speech that make up most of an exam room conversation:

Speaker A: Any chest pain with that?
Speaker B: No. Well — a little when I—
Speaker A: When you climb stairs?
Speaker B: Yes.

Get that wrong and the patient's symptom denial gets attributed to the physician's assessment. The note is then wrong in a way that's very hard to spot after the fact — which is why ambient scribe builds live or die on diarization quality. See the AI medical scribe guide for how that fits into a full pipeline.

What doesn't change

This is the part that makes Medical Mode easy to adopt, and it's worth being specific about.

No model switch. You stay on Universal-3.5 Pro or Universal-3.5 Pro Realtime. There's no separate medical model to select, no different endpoint, no different response schema.

No migration. Every other parameter you're already sending keeps working — speaker labels, PII redaction, word timings, formatting options.

No new integration. Same API, same SDKs, same webhooks. If you're already transcribing, this is a one-line diff.

Nothing to train. There's no custom vocabulary upload step, no model tuning, no dataset to assemble.

Async and streaming, side by side

Medical Mode works identically on pre-recorded and live audio. The only difference is the parameter's name — async takes speech_models as a list, streaming takes speech_model as a single string.

Pre-recorded

import assemblyai as aai

aai.settings.api_key = "YOUR_API_KEY"

config = aai.TranscriptionConfig(
    speech_models=["universal-3-5-pro"],
    domain="medical-v1",
    speaker_labels=True,
    speakers_expected=2,
)

transcript = aai.Transcriber().transcribe("visit.wav", config=config)

for u in transcript.utterances:
    print(f"Speaker {u.speaker}: {u.text}")

Streaming

import websockets

URL = (
    "wss://streaming.assemblyai.com/v3/ws"
    "?sample_rate=16000"
    "&speech_model=universal-3-5-pro"
    "&domain=medical-v1"
    "&voice_focus=far-field"
    "&mode=max_accuracy"
)

async def stream(audio_chunks):
    async with websockets.connect(
        URL, additional_headers={"Authorization": "YOUR_API_KEY"}
    ) as ws:
        async for chunk in audio_chunks:
            await ws.send(chunk)
            # Do not await ws.recv() inside the send loop; it stalls the audio stream.
            # Run a separate task that iterates: async for message in ws
            # and prints the transcript when the message type is Turn.

What differs between the two

If you can choose, choose async. The model sees the whole recording, so it resolves ambiguity using audio that hasn't happened yet from a streaming model's point of view. Use streaming when the product genuinely needs live output.

Two ways to push accuracy further

Feed the model context

Medical Mode gets you the domain. Context gets you the patient. In an internal healthcare test, feeding a patient's prior-visit note to the model as context cut missed medical terms by 31% — on top of what the domain setting already delivers. Their current medication list, their active problems, their surgeon's name: the model stops guessing at terms it's already been told to expect.

On streaming, the equivalent lever is prompt, which cut WER 10.2% across 20,000 voice agent files, with detailed context cutting medical-term entity errors 43%.

Match the acoustics

For streaming, set voice_focus to far-field when the mic is across the room and near-field for a headset or handset. Ambient scribe audio is almost always far-field, and getting this wrong costs you accuracy you paid for.

Languages

Two facts, and conflating them causes real confusion, so here they are separately.

Medical Mode covers English, Spanish, German, and French — for both pre-recorded and streaming audio. That's the domain-tuned clinical vocabulary.

The base model code-switches natively across 18 languages, with no configuration. That's general transcription accuracy on multilingual audio, including audio that changes language mid-sentence. It doesn't extend Medical Mode's clinical tuning to those other 14 languages, but it does mean a bilingual encounter doesn't fall apart.

PHI, and what the pricing covers

Medical Mode is $0.15/hr on top of the base model. Nothing else about your bill changes, and there's no minimum, no separate contract, and no per-seat component. Async lands at $0.36/hr, streaming at $0.60/hr. Current rates on pricing. If you're weighing this against per-seat dictation platforms, the medical dictation software comparison runs those numbers.

On the compliance side: AssemblyAI signs a Business Associate Addendum (BAA) for customers processing PHI and acts as a business associate under HIPAA. Supporting controls are SOC 2 Type 2, PHI redaction across both audio and transcripts, EU data residency at api.eu.assemblyai.com, and a self-hosted deployment option. See the BAA FAQ and the BAA page.

Teams already running clinical audio through these models include Sully AI, Heidi Health, Magentus Healthcare, Deepscribe, Knowtex, and Commure.

Why the one-parameter design matters more than the number

The 3.2% MER is the headline, but the thing I'd point at is the shape of the integration. Medical Mode isn't a different model, a different endpoint, or a different contract. It's a flag.

That has a consequence people don't think about until later: it means domain specialization becomes something you can vary per request rather than per deployment. The same pipeline can run medical-v1 on encounter audio and no domain at all on the front-desk small talk. You can A/B it. You can turn it on for one specialty and measure. You can add a future domain the same way, without re-architecting anything.

That's the direction this is heading — domain expertise as a per-request parameter rather than a model you have to commit to, evaluated against your own audio instead of somebody's benchmark. Which is a good reason to build your evaluation harness now, on the audio you already have, so that when the next domain ships you can measure it in an afternoon rather than a quarter.

Frequently asked questions

Do I need to migrate to a different model to use Medical Mode?

No. Medical Mode is a parameter — domain: "medical-v1" — on the models you're already using: Universal-3.5 Pro for pre-recorded audio and Universal-3.5 Pro Realtime for streaming. Same endpoint, same SDKs, same response schema, and every other parameter you send keeps working. It's a one-line change.

Does Medical Mode work with live transcription?

Yes. Set speech_model=universal-3-5-pro and domain=medical-v1 on the streaming WebSocket at wss://streaming.assemblyai.com/v3/ws. Streaming with Medical Mode is $0.60/hr combined, with turn detection defaulting to min_turn_silence 128ms and max_turn_silence 1280ms on the balanced preset, and diarization for up to 10 speakers with revision.

Which speech-to-text API is most accurate for medical terminology?

On published benchmarks, Universal-3.5 Pro with Medical Mode records the lowest Missed Entity Rate of any provider benchmarked — 3.2%. Against the base model without Medical Mode, it captures roughly 20% fewer missed medical entities and 87% fewer entity errors. Methodology is on the benchmarks page.

How does AssemblyAI accurately capture medical jargon and terminology?

Three layers, and they stack. Medical Mode tunes the model on clinical vocabulary — drugs, conditions, procedures, anatomy, dosages. Contextual prompting lets you supply patient-specific terms; feeding a prior-visit note cut missed medical terms by 31% in an internal healthcare test. And cpWER-optimized diarization makes sure the terms land with the right speaker. See solutions for medical for the full picture.

How does AssemblyAI handle HIPAA and PHI?

AssemblyAI signs a Business Associate Addendum (BAA) for customers processing PHI and operates as a business associate under HIPAA. Alongside the BAA: SOC 2 Type 2, PHI redaction across both audio and transcripts, EU data residency, and self-hosted deployment. Details are in the BAA FAQ.

What does Medical Mode cost, and does it change my base rate?

It's a $0.15/hr add-on and it doesn't change your base rate. Pre-recorded audio on Universal-3.5 Pro is $0.21/hr, so $0.36/hr with Medical Mode. Streaming on Universal-3.5 Pro Realtime is $0.45/hr, so $0.60/hr with Medical Mode. You're billed only for the audio you send with the domain set — see the docs and the medical transcription use case.

‍