What is the best speech-to-text API for an AI front desk or receptionist agent?

AssemblyAI's Universal-3 Pro Streaming is the leading speech-to-text API for AI front-desk agents — ~150ms P50 median latency, 43% better alphanumeric accuracy on names, dates, phone numbers, and insurance IDs, and dynamic keyterm prompting (up to 100 terms per session) for provider names, procedure vocabulary, and location-specific terms. Universal-3 Pro Streaming runs at $0.45/hr per stream with unlimited concurrency. For a fully managed front-desk voice agent — appointment scheduling, intake forms, caller routing, FAQ answers — the Voice Agent API ($4.50/hr) bundles STT, LLM, and TTS through a single WebSocket with sub-1-second end-to-end latency. BAA available for medical and dental practices processing PHI.

How do I build a voice agent that schedules appointments and answers caller questions?

Use the Voice Agent API for the fastest path: open a single WebSocket, set the system prompt for your office (services, providers, scheduling policies, after-hours triage), and register tools like book_appointment, reschedule, verify_insurance, and route_to_department for write-back to your EHR or practice management system. Pass provider names, common procedures, and insurance carrier names as keyterms so the model recognizes them correctly. The agent handles the full call — greeting, intake, scheduling, confirmation, SMS — through one endpoint at $4.50/hr. For BYO stacks (LiveKit, Pipecat, Twilio), use Universal-3 Pro Streaming ($0.45/hr) as the STT layer.

Does AssemblyAI support BAA for medical and dental front-office voice agents?

AssemblyAI is considered a business associate under HIPAA, and we offer a standard Business Associate Addendum (BAA) that is required under HIPAA to ensure that AAI appropriately safeguards PHI. Medical and dental practices, urgent care clinics, and behavioral health offices processing protected health information through a front-desk voice agent should execute a BAA before going live. Combine with streaming PII redaction (redact_pii=true with policies for date_of_birth, us_social_security_number, person_name, and more) so PHI is masked on finalized turns before reaching downstream systems. Contact our sales team to execute a BAA: https://www.assemblyai.com/contact/sales.

Can a voice agent verify insurance, route urgent calls, and collect intake info on the same call?

Yes. The Voice Agent API supports JSON Schema tool calling — register verify_insurance, route_urgent, and collect_intake as tools and the agent invokes them mid-conversation without dropping the caller. The agent can verify insurance against your eligibility API, route urgent symptoms to on-call staff, and collect new-patient intake (name, DOB, insurance, reason for visit) in one continuous conversation. Universal-3 Pro Streaming's 43% better alphanumeric accuracy ensures DOBs, member IDs, and phone numbers are captured cleanly the first time.

What integrations work with EHRs and practice management systems?

AssemblyAI integrates with any EHR, practice management system, or CRM that exposes a REST API or webhook — Epic, athenahealth, eClinicalWorks, Dentrix, Eaglesoft, Open Dental, Kareo, DrChrono, and custom builds. The Voice Agent API exposes tool calls (book_appointment, send_to_pms, create_patient) that the agent invokes mid-call to push structured records to your backend. For Universal-3 Pro Streaming, pipe finalized turns through the LLM Gateway (25+ models across Claude, GPT, and Gemini) for structured-field extraction before pushing to your EHR API.

Can a voice agent handle peak call volume — Monday mornings, post-holiday rushes, after-hours overflow?

Yes. Universal-3 Pro Streaming has unlimited concurrency at $0.45/hr per stream — every incoming call gets its own real-time transcription session with no queuing. The Voice Agent API scales the same way at $4.50/hr per session, so a busy multi-location practice can answer 50, 100, or hundreds of simultaneous calls during Monday morning openings or post-holiday rushes with no degradation. SOC 2 Type 2 certified; BAA available for healthcare workloads.

Solutions

Voice agents for front office

Replace hold music with instant answers. AI voice agents handle appointment scheduling, caller routing, intake forms, and FAQs — so your front desk team focuses on the patients, clients, and customers who walk through the door.

Get started free Talk to sales

Front desk agent

4 calls handled

MJ

Maria Jensen

Reschedule cleaning — Tue 2pm

Solved

DK

David Kim

Insurance verification question

Solved

RP

Rachel Park

New patient intake — Dr. Patel

Booked

TW

Tom Walsh

Confirm Fri appointment + directions

Solved

The problem

Your front desk is a bottleneck, not a welcome mat

The average medical or professional office misses 30% of inbound calls. Patients and clients hit voicemail, hang up, and book with the competitor. Staff juggle ringing phones, walk-ins, and paperwork — leading to scheduling errors, long hold times, and burned-out receptionists. Every unanswered call is a missed appointment, missed revenue, and a worse first impression. AssemblyAI gives every office an always-on AI front desk.

Built for the busiest front desks

Latency ~1s

End-to-end response — callers never wait on hold.

Entity accuracy 43%

Better alphanumeric accuracy on names, dates, phone numbers, and insurance IDs.

Coverage 24/7

Always-on for nights, weekends, holidays, and after-hours overflow.

Concurrency Unlimited

No busy signals, no hold queues — every caller answered on the first ring.

Two ways to build

Pick the API that fits your front office stack

Ship a working front-desk agent in an afternoon, or drop best-in-class streaming STT into your existing PBX or IVR.

Recommended

Voice Agent API

Our proprietary voice stack via one WebSocket. Run a managed front-desk agent that schedules, routes, and answers — zero infra to manage, ship in a single afternoon.

Best for

Appointment scheduling, intake, FAQs, caller routing
Tool calls for EHR / practice management write-back
Session resumption — recovers gracefully if the line drops
BAA available for medical and dental practices

$4.50/hr — speech, LLM, and voice all included

Get started for free

Free tier available · No credit card required

Bring Your Own Stack

Universal-3 Pro Streaming STT API

Drop best-in-class speech recognition into your existing PBX or IVR stack. Bring your own LLM and TTS for full control over routing logic and voice.

Best for

Teams running their own LLM, TTS, and PBX/IVR
~150ms P50 latency for real-time caller interaction
43% better alphanumeric accuracy on names + dates + IDs
Native LiveKit, Pipecat, Vapi, and Twilio integrations
Unlimited concurrency for peak-hour call volume

$0.45/hr — transcription only, unlimited streams

View integration docs

No concurrency caps · Autoscaling included

Caller speaks → Agent understands → Books, routes, or resolves

Caller reaches the voice agent instantly

No hold queue, no phone tree. The agent greets by name when caller ID matches your records, asks how it can help, and starts the conversation in under a second.

Transcribed with name and date accuracy

Universal-3 Pro Streaming nails proper nouns, insurance IDs, phone numbers, and appointment times — 43% fewer alphanumeric errors than alternatives. No more callbacks to verify what the caller said.

Agent takes action via tool calling

Books or reschedules appointments, routes to the right department, verifies insurance, or collects new patient intake — all mid-conversation through JSON Schema function calls.

Confirms and follows up automatically

Reads back the appointment details, sends an SMS confirmation, and pushes the structured record to the EHR, practice management system, or CRM — without staff lifting a finger.

support_agent

Front desk pipeline

Caller reaches the voice agent instantly

↓

Transcribe with name + date accuracy

↓

Take action via tool calling

↓

Confirm + follow up automatically

Quickstart

Build a front desk voice agent in minutes

Voice Agent API — front desk agent for a medical practice

# Voice Agent API: front desk agent for a medical practice
import asyncio, json, websockets

API_KEY = "YOUR_API_KEY"

async def run_front_desk():
    async with websockets.connect(
        "wss://agents.assemblyai.com/v1/ws",
        additional_headers={"Authorization": f"Bearer {API_KEY}"},
    ) as ws:
        await ws.send(json.dumps({
            "type": "session.update",
            "session": {
                "system_prompt": (
                    "You are the front desk agent for Lakewood Family Practice. "
                    "Schedule, reschedule, and cancel appointments. Route urgent "
                    "calls to on-call staff. Collect new-patient info: name, DOB, "
                    "insurance, reason for visit. Always confirm details before "
                    "booking. Never give medical advice."
                ),
                "greeting": "Thanks for calling Lakewood Family Practice — how can I help?",
                "input": {"keyterms": ["Dr. Patel", "panoramic x-ray", "deep cleaning", "Lakewood office", "Downtown office"]},
                "output": {"voice": "ivy"},
                "tools": [{
                    "type": "function",
                    "name": "book_appointment",
                    "description": "Book an appointment in the practice management system.",
                    "parameters": {
                        "type": "object",
                        "properties": {
                            "patient_name": {"type": "string"},
                            "dob": {"type": "string"},
                            "provider": {"type": "string"},
                            "appointment_time": {"type": "string"},
                            "reason": {"type": "string"},
                        },
                        "required": ["patient_name", "appointment_time"],
                    },
                }],
            },
        }))
        async for msg in ws:
            handle(json.loads(msg))  # transcript.user, reply.audio, tool.call, ...

LiveKit + Universal-3 Pro Streaming — multi-location routing

# LiveKit + AssemblyAI Universal-3 Pro Streaming: multi-location routing
from livekit.agents import Agent, AgentSession, TurnHandlingOptions
from livekit.plugins import assemblyai, cartesia, openai, silero
from livekit.plugins.turn_detector.multilingual import MultilingualModel

class FrontDeskAgent(Agent):
    def __init__(self):
        super().__init__(
            instructions=(
                "You are a front desk receptionist for a multi-location dental "
                "group. Identify which office the caller needs. Check availability "
                "across all locations. Book appointments, collect insurance details, "
                "and route complex billing questions to the office manager."
            ),
        )

async def entrypoint(ctx):
    session = AgentSession(
        stt=assemblyai.STT(
            model="u3-rt-pro",
            keyterms_prompt=["Dr. Patel", "panoramic x-ray", "deep cleaning",
                             "Lakewood office", "Downtown office"],
        ),
        llm=openai.LLM(model="gpt-4o"),
        tts=cartesia.TTS(),
        vad=silero.VAD.load(activation_threshold=0.3),
        turn_handling=TurnHandlingOptions(
            turn_detection=MultilingualModel(),
            endpointing={"min_delay": 0.5, "max_delay": 3.0},
        ),
    )
    await session.start(room=ctx.room, agent=FrontDeskAgent())

Try in Playground View full docs

Scheduling-aware accuracy

Universal-3 Pro captures dates, times, provider names, and insurance IDs with 43% fewer alphanumeric errors than alternatives — no more "was that the 13th or the 30th?" callbacks. Keyterm prompting boosts recognition of your provider list and procedure vocabulary.

Mid-call tool calling

Book appointments, check provider availability, verify insurance, and route calls — all through JSON Schema function calls that execute mid-conversation without dropping the caller. Native integrations with LiveKit, Pipecat, Twilio, and standard WebSocket clients.

Unlimited concurrent calls

Universal-3 Pro Streaming has unlimited concurrency at $0.45/hr per stream — no busy signals, no hold queues. Auto-scales for Monday morning rushes, lunch-hour spikes, and after-hours volume so every caller gets answered on the first ring.

Voice AI builders at scale on AssemblyAI

80% increase in customer satisfaction

Calabrio replaced its legacy on-premise transcription solution with AssemblyAI's API, gaining extensive language support to power its enterprise workforce and conversation intelligence platform across global markets.

Calabrio

Frequently asked questions

: AssemblyAI's Universal-3 Pro Streaming is the leading speech-to-text API for AI front-desk agents — ~150ms P50 median latency, 43% better alphanumeric accuracy on names, dates, phone numbers, and insurance IDs, and dynamic keyterm prompting (up to 100 terms per session) for provider names, procedure vocabulary, and location-specific terms. Universal-3 Pro Streaming runs at $0.45/hr per stream with unlimited concurrency. For a fully managed front-desk voice agent — appointment scheduling, intake forms, caller routing, FAQ answers — the Voice Agent API ($4.50/hr) bundles STT, LLM, and TTS through a single WebSocket with sub-1-second end-to-end latency. BAA available for medical and dental practices processing PHI.
: Use the Voice Agent API for the fastest path: open a single WebSocket, set the system prompt for your office (services, providers, scheduling policies, after-hours triage), and register tools like book_appointment, reschedule, verify_insurance, and route_to_department for write-back to your EHR or practice management system. Pass provider names, common procedures, and insurance carrier names as keyterms so the model recognizes them correctly. The agent handles the full call — greeting, intake, scheduling, confirmation, SMS — through one endpoint at $4.50/hr. For BYO stacks (LiveKit, Pipecat, Twilio), use Universal-3 Pro Streaming ($0.45/hr) as the STT layer.
: AssemblyAI is considered a business associate under HIPAA, and we offer a standard Business Associate Addendum (BAA) that is required under HIPAA to ensure that AAI appropriately safeguards PHI. Medical and dental practices, urgent care clinics, and behavioral health offices processing protected health information through a front-desk voice agent should execute a BAA before going live. Combine with streaming PII redaction (redact_pii=true with policies for date_of_birth, us_social_security_number, person_name, and more) so PHI is masked on finalized turns before reaching downstream systems. Contact our sales team to execute a BAA: https://www.assemblyai.com/contact/sales.
: Yes. The Voice Agent API supports JSON Schema tool calling — register verify_insurance, route_urgent, and collect_intake as tools and the agent invokes them mid-conversation without dropping the caller. The agent can verify insurance against your eligibility API, route urgent symptoms to on-call staff, and collect new-patient intake (name, DOB, insurance, reason for visit) in one continuous conversation. Universal-3 Pro Streaming's 43% better alphanumeric accuracy ensures DOBs, member IDs, and phone numbers are captured cleanly the first time.
: AssemblyAI integrates with any EHR, practice management system, or CRM that exposes a REST API or webhook — Epic, athenahealth, eClinicalWorks, Dentrix, Eaglesoft, Open Dental, Kareo, DrChrono, and custom builds. The Voice Agent API exposes tool calls (book_appointment, send_to_pms, create_patient) that the agent invokes mid-call to push structured records to your backend. For Universal-3 Pro Streaming, pipe finalized turns through the LLM Gateway (25+ models across Claude, GPT, and Gemini) for structured-field extraction before pushing to your EHR API.
: Yes. Universal-3 Pro Streaming has unlimited concurrency at $0.45/hr per stream — every incoming call gets its own real-time transcription session with no queuing. The Voice Agent API scales the same way at $4.50/hr per session, so a busy multi-location practice can answer 50, 100, or hundreds of simultaneous calls during Monday morning openings or post-holiday rushes with no degradation. SOC 2 Type 2 certified; BAA available for healthcare workloads.

Build an AI front desk that never misses a call

Free tier, no credit card. From scheduling to EHR write-back in an afternoon.

Get started free