New AssemblyAI-TTS-1: Text-to-speech for voice agents. Learn more
Voice agents · Customer support

Voice agents for customer support

Customer support is the highest-volume, highest-stakes use case for voice agents — and the one where accuracy failures are most expensive. Every misheard account number is a repeat call, every botched escalation a lost customer. Here's how to ship production-grade support experiences, from first-call resolution to real-time agent assist.

The production gap

Confidence vs. reality

82% of builders feel confident handling interruptions

55% of end users are frustrated by how agents handle them

The gap between shipped and actually works. Source: The Voice Agent Report.

The numbers that matter

Accuracy over cost

76%

Of organizations prioritize accuracy over cost when choosing voice AI infrastructure — because errors compound into repeat calls and churn.

Source: The Voice Agent Report

Fewer tickets

90%

Reduction in customer complaints and support tickets after Siro built on AssemblyAI.

Source: Siro case study

Phone-number error

3.55%

Universal-3.5 Pro Realtime's phone-number entity error rate on the Pipecat benchmark — best of the realtime models tested.

Source: Daily's Pipecat STT benchmark

End-of-turn

~300ms

Turn detection from tonality, pacing, and rhythm — so the agent doesn't cut callers off mid-sentence.

Source: Universal-3.5 Pro Realtime

Customer story · Report 2

“The transcription accuracy, reliability, and speed of AssemblyAI's API have greatly enhanced our operations, reinforcing our trust in their technology and solidifying our partnership.”

— Raj Shankar, SVP Product, Calabrio

The production gap

The production gap in customer support voice agents

Support agents in production face what demos never show: callers with accents and background noise, alphanumeric soup (order numbers, confirmation codes, emails), multi-turn conversations with context dependencies, and the need to escalate gracefully to a human. Accuracy failures don't stay small — they compound into repeat calls, CSAT drops, and churn.

Hears the caller, not the room

voice_focus isolates the primary speaker and suppresses background speech that would otherwise be transcribed as phantom words and fire false interruptions.

Universal-3.6 Pro Realtime

Async-grade speaker labels, live

SpeakerRevision labels speakers during the call, then corrects them with a single ~500ms revision at stream end (up to 10 speakers) — ready for supervisor dashboards.

Universal-3.5 Pro Realtime

Coaching in the moment

Live transcription feeds sentiment flags, alerts when calls go off-script, and post-call analytics via the Speech Understanding API.

Speech Understanding

Turn-based agents

IVR and call routing

For flows where you do your own turn detection and just need a finished transcript per utterance — IVR menus, call routing, one-shot voice commands — the Sync API returns a completed transcript in a single HTTP request. No WebSocket to manage.

Sync API flow

Low latency
One HTTP request
Transcript → finish the flow

Runs on Universal-3.5 Pro.

Two paths

Two paths to production support

Both run on the same Universal-3.6 Pro Realtime foundation — build the full agent on AssemblyAI, or slot the STT layer into your existing contact center stack.

Recommended

Full voice agent stack

STT, LLM, and TTS on one API key and one bill

$0.45 /hr STT

+ Text-to-Speech at $15 per 1M characters, LLM via LLM Gateway

  • Start from the LiveKit or Pipecat reference implementation
  • LLM Gateway routes the transcript to the LLM of your choice
  • Text-to-Speech API: ~90ms to first audio, 4,000+ voices
  • Order numbers and confirmation codes read back exactly as written
  • Best for: teams building new support agents
Build a support voice agent

Bring your own stack

Universal-3.6 Pro Realtime STT

The BYO layer for existing contact centers

$0.45 /hr

Transcription only — unlimited concurrency, no concurrency fees

  • Slots into LiveKit, Pipecat, Vapi, Twilio SIP
  • Entity-aware endpointing, on by default
  • Mid-stream config updates across call stages (verify → troubleshoot → escalate)
  • Native code-switching across 32 languages
  • Best for: teams with existing contact center infra
Learn more

From pilot to scale

What production actually requires

Sophisticated teams want modular infrastructure they can plug into their own pipeline. Both lanes serve this — but production has table stakes.

Production readiness

Table stakes

  • Entity accuracy Pass
  • Unlimited concurrency Pass
  • Compliance (SOC 2 / BAA) Pass
  • Escalation to human Pass
  • Exact read-back Pass
  • Entity accuracy on the numbers and names that matter. Universal-3.5 Pro Realtime posted 3.55% phone-number error on the Pipecat benchmark — best of the realtime models tested.

  • Unlimited concurrency without rate-limit surprises. Scale to peak call volume without throttling.

  • Enterprise compliance. SOC 2 Type 2, ISO 27001:2022, PCI DSS v4.0, and a BAA for healthcare-adjacent support.

  • Graceful escalation to a human. Your orchestrator (LiveKit, Pipecat) handles the transfer and passes the full transcript along.

  • voice_focus for noisy call-center floors. Background speech doesn't become phantom words.

  • Read-back callers can trust. The Text-to-Speech API says order numbers, prices, dates, and confirmation codes exactly as written.

  • Proven in production — CallRail added voice agents and Siro cut complaints and support tickets by 90%.

Bring voice agents to your contact center

Get your free API key and start building — or talk to us about an enterprise contact center deployment.

Frequently asked questions

What is the best speech-to-text API for contact centers?

AssemblyAI is built for contact-center audio: Universal-3.6 Pro Realtime is tuned for entity accuracy on credit cards, phone numbers, emails, addresses, and names, and runs with unlimited concurrency so peak call volume never hits a rate limit. Its predecessor, Universal-3.5 Pro Realtime, posted the best entity error rates of the realtime models on Daily's Pipecat benchmark (3.55% on phone numbers). Use it as a standalone STT layer in LiveKit, Pipecat, Vapi, or Twilio SIP, or pair it with LLM Gateway and the Text-to-Speech API for the full agent.

How does real-time agent assist work with a voice AI API?

Stream the live call to Universal-3.6 Pro Realtime and feed the transcript into your coaching layer: sentiment flags for frustrated callers, alerts when a conversation goes off-script, and live speaker labels from real-time speaker diarization. Post-call, the Speech Understanding API adds entity detection, topic classification, and summarization.

Can a voice agent transfer a call to a human?

Yes. Your orchestrator (LiveKit, Pipecat) handles tool calls and the transfer to a human, and passes the full transcript along so the caller never has to repeat themselves. Universal-3.6 Pro Realtime supplies the accurate transcript and turn detection that the handoff depends on.

What is the difference between a voice agent and an IVR?

An IVR follows a fixed menu tree ('press 1 for billing'); a voice agent understands natural speech and can look up an account, answer a question, or resolve an issue in conversation. For simple turn-based flows like IVR menus or call routing, the Sync API returns a finished transcript per utterance in a single HTTP request — no WebSocket to manage.

How do you keep a support call from restarting when the connection drops?

Reconnects are handled by your orchestrator. LiveKit and Pipecat re-establish the call and carry the transcript so far into the new session, so a dropped mobile call or a flaky network doesn't send the caller back to the start.

How much does it cost to run a support voice agent?

Universal-3.6 Pro Realtime is $0.45/hr with unlimited concurrent streams and no concurrency fees; keyterms prompting is included and real-time speaker diarization is an add-on. The Text-to-Speech API is $15 per 1M characters, billed on input characters. LLM cost depends on the model you route to through LLM Gateway, or you can bring your own. It all runs on one API key and one bill.