How we measure medical transcription: MER, and why WER lies to you

A speech model transcribes a 12-minute cardiology follow-up. It gets 1,180 of 1,200 words right. On paper that's a 1.7% word error rate — the kind of number that wins a bake-off. But two of the twenty misses are "Celebrex" written as "Celexa," and one is a dosage unit dropped from the end of a sentence. An NSAID has become an antidepressant, and a prescription instruction is now ambiguous. The word error rate barely moved. The clinical meaning of the note is broken.

That gap is the whole argument for measuring clinical speech recognition differently. Word error rate treats every token as equally valuable, so a model can score beautifully while fumbling exactly the words a clinician would circle in red. Missed Entity Rate — MER — measures only the tokens that carry clinical weight: drug names, conditions, procedures, anatomy, dosages, measurements.

The rankings change when you switch metrics, and they change in the same direction every time we run the benchmark. Providers that look competitive on WER separate sharply on MER. In our published benchmarks, Universal-3.5 Pro with Medical Mode enabled comes in at 3.2% MER — the lowest of any provider we've benchmarked.

Here's what MER actually measures, why it's a better proxy for clinical usability than WER, how to compute it on your own audio, and what closes the gap when a model gets medical entities wrong.

What word error rate actually measures

WER is a Levenshtein distance over words. Count the substitutions, deletions and insertions needed to turn the hypothesis into the reference, divide by the number of words in the reference. It's simple, cheap to compute, and it has been the industry's default for decades for good reason: it's a single number that correlates reasonably well with "does this transcript read correctly."

The trouble is the denominator. In a clinical encounter, most words are conversational scaffolding — "okay," "so," "and then," "how's that been feeling," "let's take a look." Those words dominate the token count. Get them all right and you can absorb a startling number of high-stakes errors before WER notices.

Every token weighted the same

WER assigns identical cost to "um" and "metoprolol." A model that reliably transcribes filler and reliably fumbles pharmacology will post a good score. That's not a flaw in the arithmetic; it's a mismatch between what the metric optimizes and what a clinical reader needs.

Punctuation and formatting noise

Normalization choices swing WER by whole percentage points. Is "150 mg" one token or two? Is "b.i.d." expanded? Is "twenty-five" the same as "25"? Two teams evaluating the same model on the same audio can report meaningfully different WERs depending on their text-normalization pipeline. Entity-level scoring is more robust here, because you're matching against a defined vocabulary rather than an arbitrary token stream.

The errors that trigger review are rare by definition

This is the part teams miss. If a documentation workflow sends a note back for human correction whenever a medication or dosage is wrong, then the review rate is driven by entity errors — a small, low-frequency slice of the transcript. Optimizing the average token accuracy does very little for that rate. You have to measure the slice directly.

What Missed Entity Rate measures

MER is the proportion of clinically meaningful entities in the reference that the model failed to produce correctly. Build a list of the medical entities that appear in the ground-truth transcript — drugs, conditions, procedures, anatomical sites, dosages, lab values, device names — then check how many survived transcription intact. Miss "amlodipine" entirely, render it as "amlodipine besylate" when the reference says otherwise, or turn it into a phonetic neighbor, and it counts as missed.

A 3.2% MER means roughly 3 in every 100 clinically significant terms need attention. That's a number a documentation lead can reason about, because it maps directly onto how much human review a workflow needs.

Why MER is harder to game

You can't improve MER by getting better at filler words. The only way to move it is to get better at the medical vocabulary itself — the long tail of brand names, generics, acronyms, and procedure names that a general-purpose model has seen comparatively little of. That makes it a more honest signal of whether a model has real domain grounding or just a good language model prior over everyday English.

The benchmark: how providers separate on MER

We benchmark medical entity accuracy across the providers healthcare teams actually evaluate. The full methodology and dataset details live on the benchmarks page; here's the shape of the result.

The interesting part isn't the ordering — it's that the ordering doesn't match what you'd predict from WER alone. A model can sit in the middle of the WER pack and near the bottom on MER, or vice versa. If you evaluate a medical transcription vendor on WER, you are effectively picking at random with respect to the thing you care about.

How to compute MER on your own audio

Vendor benchmarks are a starting point, not an answer. Your specialty mix, microphone placement, accents and background noise are yours. Here's a reproducible way to get your own number.

1. Build a reference set that looks like your worst day

Twenty to fifty encounters is enough to be directional. Include the hard cases deliberately: the noisy exam room, the masked speaker, the patient with an accent the model rarely hears, the specialty with the densest drug vocabulary. A benchmark built only from clean audio will flatter every vendor equally.

2. Annotate entities, not words

Have a clinically literate annotator mark every medication, condition, procedure, anatomical reference, dosage and measurement in the ground-truth transcript. This is the expensive step and there's no shortcut, but it's a one-time cost you amortize across every vendor evaluation and every model upgrade afterward.

3. Score with a matching rule you write down

Decide in advance how you handle generic-versus-brand equivalence, abbreviations, and partial matches on multi-word entities. Write the rule into your harness. The absolute MER matters less than comparing vendors under identical rules.

4. Run the same audio through each candidate

Enabling Medical Mode on AssemblyAI is a single parameter — no model swap, no separate endpoint:

import assemblyai as aai

aai.settings.api_key = "YOUR_API_KEY"

config = aai.TranscriptionConfig(
    speech_models=["universal-3-5-pro"],
    domain="medical-v1",
    speaker_labels=True,
)

transcriber = aai.Transcriber(config=config)
transcript = transcriber.transcribe("./encounters/cardiology-followup-01.wav")

for utterance in transcript.utterances:
    print(f"Speaker {utterance.speaker}: {utterance.text}")

Note the plural speech_models for pre-recorded audio, and domain="medical-v1" as the only thing that changes to turn Medical Mode on. Full parameter reference is in the docs.

What actually moves MER

Once you're measuring the right thing, the levers become obvious.

Domain grounding beats keyword lists

Medical Mode is a domain the model runs in, activated with domain: "medical-v1" at +$0.15/hr on top of the base rate. Against the same base model without Medical Mode, it delivers roughly 20% fewer missed medical entities and 87% fewer entity errors overall. That second number is the one worth sitting with: most of what a general model gets wrong on clinical audio isn't a coin flip, it's systematic — and systematic errors respond to domain training.

The older approach was to hand the model a list of terms you expected. That works until the list is wrong, which is most of the time in a real clinic. Domain grounding doesn't require you to predict the vocabulary in advance.

Context from the chart

Contextual prompting lets you pass relevant text alongside the audio. In an internal healthcare test, feeding a patient's prior-visit note into the request cut missed medical terms by 31%. That makes sense: the single best predictor of which drugs and conditions will come up in today's visit is what came up last time. If your application already has the chart open, you're leaving accuracy on the table by not passing it.

Getting the speakers right

An entity attributed to the wrong speaker is a different kind of error, and WER can't see it at all. Universal-3.5 Pro ships the most accurate diarization we've built, optimized for cpWER rather than DER, which means it's tuned for the case that matters clinically — short turns and overlapping speech, the "mm-hmm, and how long has that been going on" texture of a real encounter. A model that merges doctor and patient into one speaker turn produces a note where it's unclear who reported the symptom.

Multilingual encounters

The base model code-switches natively across 18 languages with no configuration, which covers the common reality of a clinician and patient moving between languages mid-sentence. Medical Mode itself is available in English, Spanish, German and French. Those are two separate facts and worth keeping straight when you scope a deployment.

Where MER fits in a real evaluation

MER shouldn't be your only metric — it should be your primary one, with WER retained as a readability check. A transcript with perfect entities and mangled grammar is still bad for a downstream summarizer. Track both; make decisions on MER.

Two other things belong in the same evaluation. First, latency, if any part of your product is live — streaming with Universal-3.5 Pro Realtime runs $0.45/hr base, with turn detection defaulting to min_turn_silence 128ms and max_turn_silence 1280ms on the balanced preset. Second, PHI handling: entity detection covers 50+ types, and redaction is available across both transcripts (+$0.08/hr) and audio (+$0.05/hr). AssemblyAI signs a Business Associate Addendum (BAA) for customers processing PHI and maintains SOC 2 Type 2. Healthcare teams including Commure, Sully AI, Heidi Health and Deepscribe build on this stack.

For the applied end of this — architecture, code, EHR handoff — see the walkthrough on building an AI medical scribe, and the evaluation guide for ambient AI scribes. Pricing for every combination is on the pricing page, and the healthcare-specific overview is at solutions/medical.

Conclusion

The metric a market optimizes for shapes the models it gets. Two decades of WER-driven leaderboards produced models that are excellent at conversational English and mediocre at pharmacology, because that's what the scoreboard rewarded. As entity-level scoring becomes the norm in clinical procurement — and it is becoming the norm — expect the next round of model releases to look different: narrower gains on general audio, much larger gains on the vocabulary that decides whether a note is safe to sign. The teams that switch their internal evaluation to MER now will be able to tell which vendors actually improved and which just got better at filler words.

Frequently asked questions

What's the difference between WER and MER in medical transcription?

WER counts every word error equally across the whole transcript. MER counts only errors on clinically meaningful entities — drugs, conditions, procedures, dosages, anatomy. A model can post a strong WER while missing a high share of medical terms, because filler and conversational words dominate the token count. MER measures the slice that determines whether a clinical note is usable.

Which speech-to-text API is most accurate for medical terminology?

On our benchmark, Universal-3.5 Pro with Medical Mode records 3.2% Missed Entity Rate — the lowest across the providers we've tested. Methodology is on the benchmarks page, and the honest answer is that you should reproduce it on your own audio before committing.

What is AssemblyAI's Medical Mode?

It's a clinical domain for the flagship models, turned on with one parameter — domain: "medical-v1" — with no model switch. It costs +$0.15/hr on top of the base rate, so $0.36/hr combined for pre-recorded audio and $0.60/hr combined for streaming. It's available in English, Spanish, German and French.

How accurate is AssemblyAI Medical Mode compared to the base model?

Against the same base model with Medical Mode off, it produces roughly 20% fewer missed medical entities and 87% fewer entity errors. Passing relevant context — a prior-visit note, for example — cut missed medical terms by a further 31% in an internal healthcare test.

Does a low MER mean I can skip human review of clinical notes?

No. A 3.2% MER is a meaningfully lower review burden than the alternatives, not zero. Clinical documentation carries liability that no accuracy figure removes, and the clinician signing the note is still the control. What better MER buys you is fewer corrections per note and less time spent on them.

How does AssemblyAI handle HIPAA and PHI?

AssemblyAI signs a Business Associate Addendum (BAA) for customers processing PHI, acting as a business associate under HIPAA. Alongside the BAA, the platform offers PHI redaction across audio and transcripts, entity detection covering 50+ types, and SOC 2 Type 2. Details are in the BAA FAQ and the BAA page.