What's the best medical transcription API?

A transcript that turns "metoprolol" into "metronidazole" isn't a typo. It's a beta blocker becoming an antibiotic, and it's sitting in a note that a downstream summarizer will happily treat as fact. That single class of error — the missed or mangled clinical entity — is what separates a medical transcription API from a general-purpose one, and it's the thing most evaluation processes measure last.

Here's the number we'd start with. Universal-3.5 Pro with Medical Mode enabled posts a 3.2% Missed Entity Rate on medical audio. That is the lowest across benchmarked providers. Same audio, same entity list, very different amounts of cleanup work landing on whoever reviews the note.

This post is about how to pick a medical transcription API on evidence rather than a feature grid: which metric actually predicts clinical usability, what the eight main providers do and don't offer, how to turn Medical Mode on in one parameter, and how PHI, BAA coverage, and data residency fit into the decision. If you're comparing two specific vendors, we also have a head-to-head on AssemblyAI vs Deepgram for medical transcription.

What a medical transcription API has to do that a general one doesn't

General speech-to-text is optimized for the average word. Clinical audio punishes that. The words that carry the most consequence — drug names, dosages, procedures, ICD-adjacent phrasing, anatomical terms — are exactly the words that are rare in general training data, phonetically crowded, and often spoken fast by someone who says them forty times a day.

So a medical transcription API has four jobs beyond "produce readable text":

• Get the entities right. Drugs, conditions, procedures, dosages. A 2% overall word error rate means nothing if the 2% is concentrated in medication names.
• Know who said what. Clinician instructions and patient-reported symptoms are different data. A transcript that merges them is a liability.
• Handle real room audio. Exam rooms have doors, sinks, keyboards, and a patient sitting four feet from the mic while a family member talks over them.
• Keep PHI handled properly. Redaction, a signed BAA, and a clear answer on where audio lives.

Everything below is organized around those four.

Missed entity rate is the metric to evaluate on

Word error rate is the default ASR benchmark and it's the wrong lens for healthcare. WER treats "the" and "hydralazine" as equally weighted tokens. Missed Entity Rate doesn't — it measures how often the model fails to correctly capture a clinical entity, which is the failure mode that actually reaches a chart.

On our medical benchmarks, Universal-3.5 Pro with Medical Mode records a 3.2% MER, the lowest across the providers we benchmarked. Against the base model without Medical Mode, that's roughly 20% fewer missed medical entities and 87% fewer entity errors. The add-on isn't a rounding-error improvement; it's a different error profile.

When you run your own eval — and you should — do it on your audio, not on clean read speech. Pull 20 to 50 real encounters, build an entity list from the notes your clinicians actually wrote, and score entity capture rather than overall WER. That's a two-day exercise that will tell you more than any vendor deck.

Medical transcription API comparison

Here's the landscape as of now. Pricing shown for AssemblyAI is list; other vendors' pricing varies by contract and region, so treat those columns as directional.

The short read: AWS and Google give you a medical endpoint bundled into a cloud you may already be in. Deepgram gives you a fast clinical model. NVIDIA Riva gives you full control and full responsibility for training. AssemblyAI's argument is that you shouldn't have to switch models or maintain a vocabulary list to get clinical accuracy — one parameter on the flagship model gets you there.

How Medical Mode works in practice

Medical Mode is a parameter, not a different model. You keep Universal-3.5 Pro and everything it does — diarization, formatting, code-switching — and add clinical entity handling on top.

Pre-recorded audio

Async transcription takes speech_models as a list:

import assemblyai as aai

aai.settings.api_key = "YOUR_API_KEY"

config = aai.TranscriptionConfig(
    speech_models=["universal-3-5-pro"],
    domain="medical-v1",
    speaker_labels=True,
)

transcript = aai.Transcriber().transcribe(
    "https://example.com/encounter-audio.wav",
    config=config,
)

for utterance in transcript.utterances:
    print(f"Speaker {utterance.speaker}: {utterance.text}")

That's $0.21/hr for the base model plus $0.15/hr for Medical Mode — $0.36/hr all in. Full detail on pricing.

Live audio

Streaming uses speech_model, singular, and connects to wss://streaming.assemblyai.com/v3/ws:

from assemblyai.streaming.v3 import (
    StreamingClient,
    StreamingClientOptions,
    StreamingParameters,
)

client = StreamingClient(
    StreamingClientOptions(api_key="YOUR_API_KEY")
)

client.connect(
    StreamingParameters(
        sample_rate=16000, speech_model="universal-3-5-pro",
        domain="medical-v1",
        voice_focus="far-field",
        mode="balanced",
        speaker_labels=True,
    )
)

Universal-3.5 Pro Realtime is $0.45/hr base, $0.60/hr with Medical Mode. Turn detection defaults to min_turn_silence 128ms and max_turn_silence 1280ms on the balanced preset, which is what makes it usable inside a live conversation rather than as a delayed caption track.

Contextual prompting

This is the feature we'd point at first if you're building anything longitudinal. You can feed the model context before it transcribes — and in a clinical setting, the most useful context is the patient's own prior-visit note. In an internal healthcare test, doing exactly that cut missed medical terms by 31%.

Think about why. A patient on lisinopril and metformin who was seen for the same complaint six weeks ago has a vocabulary. Handing the model that vocabulary as free text — not a curated keyword list you have to maintain — resolves most of the phonetic ambiguity before it becomes an error. The streaming equivalent is agent_context, which cut WER 10.2% across 20,000 voice agent files, and detailed context cut medical-term entity errors 43%.

Doctor, patient, and the third person in the room

Diarization is where a lot of clinical transcription quietly falls apart. The failure isn't usually a long monologue attributed to the wrong person — it's the short turns. "Mm-hm." "How long?" "Since Tuesday." Those two-word exchanges carry the clinical content, and models tuned for diarization error rate tend to smooth right over them.

Universal-3.5 Pro's diarization is optimized for cpWER rather than DER, which means it's scored on getting the right words attributed to the right speaker rather than on segmenting the timeline neatly. It's the most accurate diarization we've shipped, and it holds up on short turns and overlapped speech — the two things exam-room audio is made of. In streaming, diarization supports revision as more audio arrives, up to 10 speakers.

For far-field ambient capture, voice_focus has near-field and far-field settings. Set it to match your hardware. A phone on a desk three feet from the clinician and a headset mic are not the same acoustic problem.

Multilingual clinical audio

Two separate facts here, and they get conflated constantly.

First: Universal-3.5 Pro code-switches natively across 18 languages, no configuration. A patient who moves between English and Spanish mid-sentence — the normal case in a lot of clinics — doesn't break the transcript or require you to detect and route language up front.

Second: Medical Mode itself supports English, Spanish, German, and French, for both pre-recorded and streaming audio. So the clinical entity layer covers four languages while the underlying model handles considerably more general speech. If your patient population needs clinical entity accuracy in a fifth language, that's a roadmap conversation, not a config flag.

PHI, BAA, and where the audio lives

Compliance shouldn't be the reason you pick a vendor, but it will absolutely be the reason procurement blocks one.

AssemblyAI signs a Business Associate Addendum (BAA) for customers processing PHI, acting as a business associate under HIPAA. Details are in our BAA FAQ and the Business Associate Agreement itself. We're SOC 2 Type 2.

On the data side, PHI redaction runs across both audio and transcripts — the identifiers are removed from the text and bleeped in the media file, so you're not left with a clean transcript sitting next to an unredacted recording. For teams with residency requirements, there's EU data residency via api.eu.assemblyai.com, and a self-hosted deployment option when the audio can't leave your infrastructure at all.

The practical sequencing advice: get the BAA conversation started in parallel with your technical eval, not after it. It's rarely the blocker people expect, but it's always the one that adds weeks if you serialize it.

Voice agents and the healthcare front desk

Transcription is one half of the healthcare voice story. The other is the phone line — appointment scheduling, refill requests, intake, triage routing — where you need speech-to-text, an LLM, and text-to-speech stitched together with latency low enough that the caller doesn't start talking over the bot.

The Voice Agent API collapses that stack into one WebSocket at a flat $4.50/hr, replacing separate STT, LLM, and TTS vendors and the orchestration between them. For healthcare specifically, that means the entity accuracy that matters for a refill request ("Is that amoxicillin or amoxicillin-clavulanate?") comes from the same model family you'd use for clinical documentation.

Our partners have been vocal about the context handling in particular:

"We're excited to make AssemblyAI's Universal-3.5 Pro available on LiveKit Inference. What really stands out is their pace of innovation with Context Carryover — it intelligently applies conversation context to improve transcription accuracy in a way most speech models don't, removing the need for users to predefine key terms." — David Zhao, Co-founder at LiveKit

That last clause is the one healthcare teams should notice. Maintaining a keyword list per specialty, per formulary, per clinic is a real operational cost. Not having to is a feature.

How to actually decide

A decision framework that fits on a napkin:

• Already deep in AWS or GCP with modest accuracy requirements? Their medical endpoints are the path of least resistance. Just measure the MER before you commit.
• Building an ambient scribe? Entity accuracy and diarization quality dominate everything else. See our guide to choosing a speech-to-text API for ambient scribes.
• Replacing a general model that's failing on drug names? Medical Mode is one parameter. That's a one-afternoon test.
• Hard data residency constraints? Start with self-hosted and EU residency options and narrow from there.
• Comparing across the whole field? Our roundup of the best medical speech-to-text options goes provider by provider.

Teams building on this stack today include Commure, Sully AI, Heidi Health, Deepscribe, Knowtex, and Magentus Healthcare — a mix of ambient scribes, clinical workflow tools, and health system deployments. Different products, same underlying requirement: get the entities right.

Where this goes next

The interesting shift isn't accuracy — accuracy is a solved-enough problem that the top of the benchmark is getting crowded. It's that context is becoming the primary lever. Contextual prompting means the model's performance on a given encounter now depends on what you tell it beforehand, which turns transcription from a stateless API call into something closer to a personalized service. The 31% reduction in missed terms from a prior-visit note is a preview of that: the same model, the same audio, materially better output, because it knew who it was listening to.

Which means the teams that win the next round won't be the ones who picked the best model. They'll be the ones who built the plumbing to feed it the right context at the right moment — patient history, care setting, specialty, prior encounters. Start designing for that now, even if you're only using it for a keyword hint today.

Frequently asked questions

What is the best speech-to-text API for medical transcription?

On published medical benchmarks, Universal-3.5 Pro with Medical Mode has the lowest Missed Entity Rate at 3.2%. See the full benchmarks. That said, "best" depends on your audio — run the same 20 to 50 encounters through your shortlist and score entity capture, not overall word error rate.

What is AssemblyAI's Medical Mode?

It's a clinical accuracy layer you enable with one parameter, domain: "medical-v1", on top of Universal-3.5 Pro or Universal-3.5 Pro Realtime. No model switch, no vocabulary list to maintain. It costs $0.15/hr on top of the base rate, and it delivers roughly 20% fewer missed medical entities and 87% fewer entity errors than the base model alone.

How accurate is AssemblyAI Medical Mode compared to other providers?

3.2% Missed Entity Rate, the lowest across the providers we benchmarked, which include Deepgram, Speechmatics, AWS, and Google.

How does AssemblyAI handle HIPAA and PHI?

AssemblyAI signs a Business Associate Addendum (BAA) for customers processing PHI and operates as a business associate under HIPAA. We're also SOC 2 Type 2, offer PHI redaction across both audio and transcripts, and support EU data residency and self-hosted deployment. See our BAA FAQ for how to get one in place.

Does AssemblyAI automatically redact patient PII from medical transcripts?

Yes — PHI redaction runs across both the transcript text and the audio itself, so identifiers are removed from the text and obscured in the media file. That matters because a redacted transcript paired with an unredacted recording isn't actually redacted.

How does AssemblyAI capture medical jargon and terminology accurately?

Three mechanisms. Medical Mode adds a clinical entity layer trained on drugs, conditions, and procedures. Contextual prompting lets you pass free-text context — a patient's prior-visit note cut missed medical terms by 31% in an internal healthcare test. And the base model's native code-switching across 18 languages means multilingual encounters don't degrade the terminology handling. Details in the docs and on our medical solutions page.

‍