What is the best speech-to-text API for building an ambient AI scribe?

For clinical ambient scribes the deciding factor is medical entity accuracy, not raw word error rate. Universal-3.5 Pro with Medical Mode posts a 3.2% Missed Entity Rate — the lowest across benchmarked providers including Deepgram, Speechmatics, AWS, and Google — and runs on pre-recorded and realtime audio with one parameter.

How do I add Medical Mode to an ambient scribe pipeline?

Set domain: "medical-v1" on Universal-3.5 Pro or Universal-3.6 Pro Realtime. No model switch, no new API key. It's a $0.15/hr add-on billed per second.

Can an ambient AI scribe transcribe a patient visit in real time?

Yes. Medical Mode works on Universal-3.6 Pro Realtime (307 ms median time to final transcript, Pipecat open benchmark), so you can draft the note as the conversation happens.

How much more accurate is Medical Mode than the base speech-to-text model?

About 20% fewer missed medical entities and 87% fewer entity errors than the base model without Medical Mode, measured on clinical audio.

Does AssemblyAI redact patient information (PHI) from medical transcripts?

Yes — PHI redaction across audio and transcripts. AssemblyAI is a business associate under HIPAA and signs a standard BAA for customers processing PHI; SOC 2 Type 2.

What languages does Medical Mode support for clinical transcription?

English, Spanish, German, and French, on pre-recorded and realtime.

Ambient AI scribes

The speech layer ambient scribes are built on

Ambient scribes live or die on the words they miss. Universal-3.5 Pro with Medical Mode reaches a 3.2% Missed Entity Rate on clinical conversations — about 20% fewer missed medical entities than the base model without Medical Mode — so the note your product drafts starts from a transcript that already has the drugs, dosages, and conditions right. Add it with one parameter, on async or realtime.

Try Medical Mode free on your audio Read the docs

Built on the model with the lowest Missed Entity Rate across benchmarked providers

Missed Entity Rate 3.2%

Missed Entity Rate on clinical conversations with Medical Mode.

vs. base model ~20%

Fewer missed medical entities than the base model without Medical Mode.

vs. base model 87%

Fewer medical entity errors than the base model.

Benchmarked providers #1

Lowest Missed Entity Rate vs. Deepgram, Speechmatics, AWS, and Google.

See the medical benchmarks

Activation

One parameter on top of your existing pipeline

No model swap, no re-integration. If you already transcribe with AssemblyAI, ambient Medical Mode is a one-line change — and it runs on both pre-recorded and realtime.

import assemblyai as aai

aai.settings.api_key = "YOUR_API_KEY"

# Ambient scribe: full-visit audio -> clinical-grade transcript
config = aai.TranscriptionConfig(
    speech_models=["universal-3-5-pro", "universal-2"],
    domain="medical-v1",
    keyterms_prompt=["metoprolol", "tachycardia", "hyperlipidemia", "SOAP note"],
)

transcript = aai.Transcriber().transcribe("./clinic-visit.wav", config)
print(transcript.text)

Entities you can build a note on

Medications, dosages, conditions, and procedures land in the transcript, so your note-generation layer isn't correcting the source before it can summarize.

Universal-3.5 Pro

Async or ambient realtime

Run on uploaded recordings or live during the encounter on Universal-3.6 Pro Realtime with a 307 ms median time to final transcript on Pipecat's open benchmark.

Universal-3.6 Pro Realtime

Built for production scale

Millions of hours, unlimited concurrency, pay-as-you-go at $0.15/hr on top of Universal-3.5 Pro.

Medical Mode

Teams building ambient scribes on AssemblyAI

"In the medical context, accuracy is highly important. The biggest impact has been enabling our technical team to focus on workflow-specific features rather than a general speech-to-text pipeline."

Jackson Bierfeldt · Cofounder & CTO, JotPsych

Common questions

: For clinical ambient scribes the deciding factor is medical entity accuracy, not raw word error rate. Universal-3.5 Pro with Medical Mode posts a 3.2% Missed Entity Rate — the lowest across benchmarked providers including Deepgram, Speechmatics, AWS, and Google — and runs on pre-recorded and realtime audio with one parameter.
: Set domain: "medical-v1" on Universal-3.5 Pro or Universal-3.6 Pro Realtime. No model switch, no new API key. It's a $0.15/hr add-on billed per second.
: Yes. Medical Mode works on Universal-3.6 Pro Realtime (307 ms median time to final transcript, Pipecat open benchmark), so you can draft the note as the conversation happens.
: About 20% fewer missed medical entities and 87% fewer entity errors than the base model without Medical Mode, measured on clinical audio.
: Yes — PHI redaction across audio and transcripts. AssemblyAI is a business associate under HIPAA and signs a standard BAA for customers processing PHI; SOC 2 Type 2.
: English, Spanish, German, and French, on pre-recorded and realtime.

Try Medical Mode free on your audio

Run your own clinical audio through Medical Mode in the playground — one parameter, no model swap. Free to start, no credit card required.

Try Medical Mode free