How accurate are AI transcripts for technical or medical terms?

A cardiologist says "hydralazine." The transcript says "hydroxyzine." One is a vasodilator, the other an antihistamine. Every other word in the sentence is correct — the dose, the frequency, the route — so a word error rate calculation would score that transcript at better than 99% accurate. The one word that carried clinical meaning is wrong, and a downstream summarizer has no way to know it.

That mismatch is the whole problem with judging transcription quality by word error rate alone. In our own benchmarking, Universal-3.5 Pro with Medical Mode enabled posts a 3.2% Missed Entity Rate — meaning it misses about three of every hundred clinically meaningful entities in the audio. That's 87% fewer entity errors than the same model running without Medical Mode. The base model's word error rate barely moves between those two runs. The entity accuracy moves enormously.

This post is about the mechanics: why rare technical vocabulary is genuinely hard for speech models, what actually fixes it, and how to configure a transcription pipeline so drug names, procedure codes, and part numbers survive the trip. If you're looking for the measurement side — how MER is calculated, what benchmarks to trust, and how providers stack up on published numbers — that's the companion piece on AI medical transcription accuracy.

Why technical vocabulary is the hardest part of speech recognition

General-purpose speech models are trained to predict the most likely next word. That's exactly the behavior you want for conversational audio and exactly the behavior that destroys specialist terminology.

Rare words lose to common neighbors

"Metoprolol" appears vanishingly rarely in general audio. "Metropolis" appears often. When the acoustic signal is ambiguous — and in real clinical audio it usually is — a language model weighted toward everyday speech picks the frequent word. The same dynamic turns "Xarelto" into "Zarelda," "sertraline" into "certain line," and "PCI" into "PC I."

Drug names are designed to be distinctive on paper, not in speech

Pharmaceutical naming avoids look-alike spellings. It does much less to avoid sound-alike pronunciations. Clonidine and klonopin. Hydralazine and hydroxyzine. Cephalexin and ciprofloxacin under a fast speaker. These pairs are separated by one or two phonemes, and those phonemes are the ones that vanish first when someone talks over a ventilator or a hallway PA system.

Clinical speech is compressed and heavily abbreviated

Clinicians speak in shorthand: "pt afebrile, CBC unremarkable, cont ASA 81, f/u two weeks." A model that has learned fluent English wants to expand and smooth that. It inserts articles, guesses at expansions, and normalizes numbers into forms that break structured extraction downstream.

The audio itself is usually bad

Exam rooms have hard surfaces and running equipment. Telehealth calls arrive through consumer laptop microphones at variable bitrates. Ambient recordings put the microphone on a desk while the clinician moves around the room and the patient sits three feet further away. Terminology errors cluster exactly where signal-to-noise ratio drops, which is why an accuracy number measured on clean studio audio tells you very little about what you'll see in production.

Two people talk at once

A patient starts describing a symptom while the clinician is still finishing a question. Most diarization systems either drop the overlapped segment or attribute it to the wrong speaker. If the overlapped words include a medication the patient is already taking, that's a documentation error with a clinical consequence.

Word error rate hides the errors that matter

Word error rate counts substitutions, insertions, and deletions across every token equally. In a 200-word clinical exchange, "the" and "amiodarone" carry identical weight. That makes WER useful for tracking general model quality and nearly useless for deciding whether a transcript is safe to put in front of a physician.

What Missed Entity Rate measures instead

Missed Entity Rate looks only at the tokens that carry meaning: drugs, dosages, conditions, procedures, anatomy, negations. A transcript that gets every function word right and fumbles two medication names scores well on WER and badly on MER. MER is the number we publish for Medical Mode because it's the number that predicts whether a clinical note needs rework.

Diarization needs its own metric

Speaker attribution has the same measurement trap. Diarization error rate scores how much audio was assigned to the right speaker, which rewards a system that gets the long monologues right and misses every short interjection. We optimize Universal-3.5 Pro's diarization for cpWER — concatenated minimum-permutation word error rate — because that metric only improves if the words themselves land under the correct speaker. It's the most accurate diarization we've shipped, and the gain shows up most on short turns and overlapped speech, which is precisely where clinical encounters live.

You can see the full methodology and the provider-by-provider numbers on our benchmarks page.

What actually fixes technical terminology

Three mechanisms move the needle, and they stack.

Medical Mode: one parameter, no model switch

Medical Mode is a domain adaptation you turn on with a single field — domain: "medical-v1" — on top of Universal-3.5 Pro. You don't change models, don't manage a second endpoint, and don't maintain a vocabulary list. It shifts the model's decoding priors toward clinical language, which is why the entity numbers move so much more than the word error rate does: the model stops preferring the common everyday word when a rare clinical one is acoustically plausible.

Enabled, it delivers roughly 20% fewer missed medical entities than the base model, 87% fewer entity errors, and a 3.2% absolute Missed Entity Rate — the lowest across the providers we benchmarked.

import assemblyai as aai

aai.settings.api_key = "YOUR_API_KEY"

config = aai.TranscriptionConfig(
    speech_models=["universal-3-5-pro"],
    domain="medical-v1",
    speaker_labels=True,
)

transcript = aai.Transcriber(config=config).transcribe("clinic-visit.wav")

for utterance in transcript.utterances:
    print(f"Speaker {utterance.speaker}: {utterance.text}")

That's the entire integration. Async transcription runs at $0.21/hr, Medical Mode adds $0.15/hr, so the combined rate is $0.36/hr — see pricing for the full breakdown.

Contextual prompting: tell the model what it's about to hear

Domain adaptation gets you general clinical language. Contextual prompting gets you this patient. Passing a patient's prior-visit note alongside the audio cut missed medical terms by 31% in an internal healthcare test — because the model now has the specific medication list, the specific diagnoses, and the specific provider names in front of it before it starts decoding.

This is the single biggest accuracy win most teams haven't taken. It replaces the old pattern of maintaining a static custom vocabulary file that goes stale, and it scales to any specialty without per-specialty engineering. If your system already has the chart open when the recording starts, the context is free.

Speaker separation, so terminology lands on the right person

"I stopped taking the lisinopril" means something completely different depending on who said it. Turning on speaker_labels with Universal-3.5 Pro gives you attribution that holds up through interruptions and crosstalk, up to 10 speakers in streaming with revision as more audio arrives.

Multilingual clinical audio

Universal-3.5 Pro code-switches natively across 18 languages with no configuration — a patient switching from English to Spanish mid-sentence doesn't need a language hint or a second request. Medical Mode itself covers English, Spanish, German, and French. Those are two separate capabilities and it's worth keeping them straight: the base model's language range is wider than Medical Mode's, so a Portuguese encounter transcribes well but doesn't get clinical domain adaptation.

How the providers compare on technical terminology

The pattern in that table matters more than any single row. Most providers treat specialist vocabulary as a list-maintenance problem: you tell the system which words to expect, and accuracy tracks how complete your list is. That works until you meet a drug you didn't list. Domain adaptation plus record-derived context works on vocabulary nobody enumerated in advance, which is the realistic case.

Making bad audio survivable

Terminology accuracy and audio quality are the same problem wearing different clothes. A few things help disproportionately.

Capture at 16 kHz or better and skip aggressive lossy compression on the upload path. Put the microphone closer to the speakers than to the equipment. If you're recording ambient encounters, Universal-3.5 Pro Realtime's voice_focus setting has near-field and far-field profiles — far-field is the right choice for a desk or ceiling microphone in a room, and picking the wrong one costs you real accuracy on quiet speakers.

For live pipelines, the same terminology tools apply. Streaming uses the singular speech_model field and the v3 WebSocket endpoint:

import websockets
from urllib.parse import urlencode

WS_URL = "wss://streaming.assemblyai.com/v3/ws"

config = {
    "speech_model": "universal-3-5-pro",
    "domain": "medical-v1",
    "sample_rate": 16000,
    "voice_focus": "far-field",
    "mode": "max_accuracy",
    "speaker_labels": True,
}

async def stream(audio_chunks):
    async with websockets.connect(
        WS_URL + "?" + urlencode({k: str(v).lower() if isinstance(v, bool) else v for k, v in config.items()}), additional_headers={"Authorization": "YOUR_API_KEY"}
    ) as ws:
        # Config travels in the query string above; v3 has no configure message
        async for chunk in audio_chunks:
            await ws.send(chunk)
            # Do not await ws.recv() inside the send loop; it stalls the audio stream.
            # Run a separate task that iterates: async for message in ws
            # and prints the transcript when the message type is Turn.

Streaming with Universal-3.5 Pro Realtime runs $0.45/hr base, $0.60/hr with Medical Mode. The mode field trades latency against accuracy across min_latency, balanced, and max_accuracy — for documentation workloads where nobody is waiting on a response, choose accuracy. For conversational agents, turn detection defaults to min_turn_silence 128ms and max_turn_silence 1280ms on the balanced preset, and prompt cut word error rate 10.2% across 20,000 voice agent files, with detailed context cutting medical-term entity errors 43%.

Where technical accuracy shows up outside medicine

The same failure mode appears anywhere vocabulary is dense and consequential. Legal audio has case citations and party names. Financial calls have ticker symbols and instrument names that sound like ordinary words. Manufacturing and field service audio is full of part numbers that a general model happily converts into English phrases. Insurance claims audio mixes medical terminology with policy language.

In each case the fix is the same shape: pick a model that handles rare tokens well, give it whatever document context you already have, and measure on the entities you care about rather than on aggregate word accuracy. Teams building on AssemblyAI's speech-to-text across healthcare — Sully AI, Heidi Health, Deepscribe, Knowtex, Magentus Healthcare — converge on that pattern regardless of specialty.

Handling PHI once the transcript exists

Accuracy work produces a document full of protected health information, and that document needs the same care as the audio. AssemblyAI signs a Business Associate Addendum (BAA) for customers processing PHI, operates as a business associate under HIPAA, and offers PHI redaction across both audio and transcripts so identifiers can be removed from the artifacts you retain. The platform is SOC 2 Type 2 audited, and self-hosted deployment plus EU data residency are available where residency requirements apply. The details live in the BAA FAQ and our BAA terms.

What changes next

The interesting shift isn't better acoustic modeling — it's that the model no longer has to guess in isolation. Contextual prompting turned transcription from a pure signal-processing task into one that reads the chart first, and the 31% reduction in missed terms came from information the health system already had sitting in a database. As more of that context flows into the request — the problem list, the active prescriptions, the referral note, the imaging report — the remaining error surface shrinks toward genuinely novel vocabulary. The teams that win on clinical accuracy over the next two years won't be the ones with the biggest custom dictionaries. They'll be the ones who wired their record system into the transcription call.

Frequently asked questions

Which speech-to-text API is most accurate for medical terminology?

On published Missed Entity Rate, Universal-3.5 Pro with Medical Mode leads the providers we benchmarked at 3.2% MER. Methodology and the full comparison are on our benchmarks page. Run your own audio through before committing — provider rankings on generic benchmarks don't always survive contact with your specific accents, specialties, and recording setup.

How does AssemblyAI capture medical jargon and terminology accurately?

Three layers. Medical Mode (domain: "medical-v1") adapts the flagship model's decoding toward clinical language, cutting entity errors 87% versus the base model. Contextual prompting lets you pass a patient's prior-visit note with the audio, which cut missed medical terms 31% in an internal test. And cpWER-optimized diarization keeps terminology attached to the right speaker through interruptions.

How do I improve transcription of poor-quality audio?

Fix capture first — 16 kHz or higher, minimal lossy compression, microphone closer to people than to equipment. Then set voice_focus to far-field for room recordings or near-field for headsets, and use max_accuracy mode when latency isn't critical. Contextual prompting helps more on bad audio than on clean audio, because that's where the model most needs a prior.

Can I inject custom vocabulary instead of using Medical Mode?

You can supply context, and we'd recommend it — but as a complement, not a replacement. A static word list only covers terms you predicted. Medical Mode changes how the model weighs rare clinical vocabulary generally, so it helps on drugs and procedures nobody added to a list. Using both together is the strongest configuration.

Does accuracy hold up in languages other than English?

Universal-3.5 Pro code-switches natively across 18 languages with no configuration, so mixed-language encounters transcribe without a language hint. Medical Mode's clinical adaptation covers English, Spanish, German, and French. Outside those four you still get strong general transcription, just without the clinical domain boost.

How does AssemblyAI handle HIPAA and PHI?

AssemblyAI signs a Business Associate Addendum (BAA) for customers processing PHI and operates as a business associate under HIPAA. On top of that: PHI redaction across audio and transcripts, SOC 2 Type 2 audited infrastructure, and self-hosted or EU-resident deployment where you need it. See the BAA FAQ for how to request one, or our healthcare solutions page for the broader picture.

‍