customers
All customer stories
Top Voice AI companies are building with Assembly.
resources
Latest Release
Dictation API
The first API built for dictation. Your users speak, and it returns text that is ready to send: filler gone, self-corrections resolved, names spelled right.
resources
A healthcare voice agent holds a spoken conversation with a patient — triage, intake, scheduling — and acts on what it hears. AssemblyAI supplies the speech layer: Universal-3.6 Pro Realtime with Medical Mode holds a 3.2% Missed Entity Rate on clinical conversations, the lowest across benchmarked providers. BAA available.
Automate scheduling, insurance verification, and patient intake with AI voice agents powered by the most accurate medical speech-to-text. Build end-to-end with our Voice Agent API, or drop Universal-3.6 Pro Realtime with Medical Mode into your existing stack.
Patients abandon long IVR menus and fill paper intake forms that should take seconds. Generic speech-to-text mangles medication names, dosages, and ICD references — and PHI handling is an afterthought instead of a primitive. Modern healthcare voice agents — built on medical-grade streaming STT with in-stream PHI redaction, a managed LLM, and natural TTS — handle patient intake, telehealth triage, appointment scheduling, and prescription refills before a human staff member ever picks up. They replace legacy IVR, free clinical staff from low-acuity calls, and capture structured data into the EHR in the same conversation.
Fewer medical entity errors with Medical Mode for pharma, anatomy, and ICD references.
Median time to final transcript on Universal-3.6 Pro Realtime (Pipecat open benchmark).
Business Associate Addendum available for covered entities processing PHI. SOC 2 Type 2 certified.
Medical Mode covers English, Spanish, German, and French.
Two ways to build
Ship a working intake or triage agent in an afternoon, or drop medical-grade STT into the orchestrator you already run.
Our proprietary voice stack via one WebSocket. Connect, stream patient audio in, get audio back — we handle the rest.
Best for
Best-in-class healthcare voice agents — the preferred way to build with AssemblyAIPatient intake, telehealth triage, prescription refills, appointment scheduling, post-discharge follow-upTeams shipping fast — working agent in an afternoon, no infra to manageBusiness Associate Addendum (BAA) available, in-stream PHI redactionFree tier available · No credit card required
The STT layer for your cascading voice agent architecture. Works natively with LiveKit, Pipecat, and Twilio.
Best for
Teams already using LiveKit, Pipecat, or Vapi as their orchestration layerCascading architectures (STT → LLM → TTS) with EHR tool callsHigh-scale deployments where margin and full control matterWorkflows with RAG over clinical notes or proprietary LLMsBAA-eligible, SOC 2 Type 2 — bring your own compliance infrastructureNo concurrency caps · Autoscaling included
Ingest patient audio
Voice Agent API: single WebSocket. Or Twilio Media Streams / SIP → U3.6 Pro Realtime for BYO stack.
Clinical-grade transcription
Universal-3.6 Pro Realtime with Medical Mode + keyterm prompting for medications, anatomy, and ICD codes.
PHI redaction in-stream
Names, DOBs, MRNs, addresses, and account IDs masked before transcripts leave the pipeline.
LLM reasoning + EHR tool calls
Triage rules, scheduling, refills, intake — managed (Voice Agent API) or BYO with LiveKit / Pipecat.
Voice response
TTS streamed back to patient. Full round-trip under 1 second with natural turn detection.
Voice Agent API — recommended
# Voice Agent API — single WebSocket, full healthcare pipeline
import asyncio, json, websockets
API_KEY = "YOUR_API_KEY"
async def run_agent():
async with websockets.connect(
"wss://agents.assemblyai.com/v1/ws",
additional_headers={"Authorization": f"Bearer {API_KEY}"},
) as ws:
await ws.send(json.dumps({
"type": "session.update",
"session": {
"system_prompt": (
"You are a triage agent for Acme Health. "
"Collect symptoms, medications, and insurance."
),
"greeting": "Hi, this is Acme Health — what brings you in today?",
"output": {"voice": "ivy"},
},
}))
async for msg in ws:
handle(json.loads(msg)) # session.ready, transcript.user, reply.audio, tool.call, ...
Universal-3.6 Pro Realtime + LiveKit — BYO stack
# LiveKit + AssemblyAI Universal-3.6 Pro Realtime — healthcare voice agent
from livekit.agents import Agent, AgentSession, TurnHandlingOptions
from livekit.plugins import assemblyai, cartesia, openai, silero
class IntakeAgent(Agent):
def __init__(self):
super().__init__(
instructions=(
"You are a patient intake agent for Acme Health. "
"Collect symptoms, current medications, allergies, and insurance."
),
)
async def entrypoint(ctx):
session = AgentSession(
stt=assemblyai.STT(
model="universal-3-6-pro",
min_turn_silence=100,
max_turn_silence=1000, # patients pause longer; bump higher for entity dictation
vad_threshold=0.3,
keyterms_prompt=["amoxicillin", "metformin", "ICD-10", "PCP referral"],
),
llm=openai.LLM(model="gpt-4o"),
tts=cartesia.TTS(),
vad=silero.VAD.load(activation_threshold=0.3), # match AssemblyAI's vad_threshold
turn_handling=TurnHandlingOptions(
turn_detection="stt",
endpointing={"min_delay": 0}, # AssemblyAI handles endpointing; avoid additive delay
),
)
await session.start(room=ctx.room, agent=IntakeAgent())
Universal-3.6 Pro Realtime with Medical Mode reduces medical entity errors by 87% — pharmaceutical names, anatomical terms, dosages, and ICD references come through clean on phone and far-field audio.
PHI redaction runs in-stream and is configurable per session. BAA available on request, with SOC 2 Type 2, AES-256 encryption, and optional EU data residency.
Accurately separate patient, provider, and staff turns even when speakers move in and out of frame — critical for telehealth visits, rounds, and ambient documentation.
JotPsych uses AssemblyAI to power ambient clinical documentation for mental health providers — capturing nuanced patient conversations accurately and securely.
We're excited to make AssemblyAI's Universal-3.5 Pro available on LiveKit Inference. What really stands out is their pace of innovation with Context Carryover — it intelligently applies conversation context to improve transcription accuracy in a way most speech models don't, removing the need for users to predefine key terms.
David Zhao · Co-founder, LiveKit
Missed Entity Rate is the share of clinical entities — medication names, dosages, diagnoses — that a model fails to transcribe correctly. Lower is better.
| Provider | Missed Entity Rate | Word Error Rate |
|---|---|---|
| AssemblyAI Universal-3.5 Pro w/ Medical Mode | 3.2% | 5.3% |
| Speechmatics Enhanced Medical | 3.6% | 5.5% |
| Deepgram Nova-3 Medical | 4.7% | 6.1% |
| AWS Transcribe Medical | 8.7% | 5.9% |
| Google Medical Conversation | 24.4% | 12.9% |