New AssemblyAI-TTS-1: Text-to-speech for voice agents. Learn more
Insights & Use Cases

Best speech-to-speech voice agent APIs in 2026, compared with the cascaded alternative

A vendor roundup of the speech-to-speech and realtime voice agent options in 2026: OpenAI Realtime and GPT-Live-1, Gemini Live, ElevenLabs, Telnyx and Vapi. Compare accuracy, latency, pricing and control, see when a chained pipeline on Universal-3.6 Pro Realtime is the better call, and pick one with a 5-step evaluation.

Abstract green cylinder illustration

Written by

Kelsey Foster

Published on

3 November 2026

The best speech-to-speech voice agent APIs in 2026 are OpenAI's GPT-Live-1 and gpt-realtime-2.1, Google's Gemini 3.8 Live, and Amazon Nova 2 Sonic. They take caller audio in and send agent audio back over one connection. Managed agent APIs like ElevenLabs ElevenAgents and the Deepgram Voice Agent API look similar from the outside, but under the hood they run a cascade of speech-to-text, an LLM and text-to-speech. And if your agent has to get names, codes and phone numbers right, the strongest option is a cascaded pipeline you control: streaming speech-to-text, an LLM, and text-to-speech, wired together by your orchestrator.

As of November 2026:

  • Native speech-to-speech: OpenAI GPT-Live-1 ($0.05 per minute plus backend model usage), gpt-realtime-2.1, Gemini 3.8 Live (70 languages) and Amazon Nova 2 Sonic (8-minute connection limit).
  • Managed cascades: ElevenAgents and the Deepgram Voice Agent API bundle STT, LLM and TTS behind one API.
  • Cascaded pipeline you own: Universal-3.6 Pro Realtime (5.19% mean WER and 14.41% entity error on AssemblyAI's voice-agent benchmark), any LLM through LLM Gateway, and AssemblyAI's Text-to-Speech API at $15 per 1M characters.

Sub-second latency is table stakes now. What separates the options is accuracy on real phone audio, how much of each layer you control, and pricing you can forecast at scale. In our voice agent insights report, 76% of builders ranked speech-to-text accuracy as their top non-negotiable. This roundup compares the speech-to-speech and realtime options vendor by vendor, shows where a cascaded pipeline beats them, and gives you a 5-step evaluation to pick one.

Speech-to-speech vs. cascaded: the short version

A native speech-to-speech model processes audio end to end. One model hears, reasons and speaks. It can sound expressive, but you get less visibility into what it heard and less control over each layer.

A cascaded (or chained) pipeline runs streaming STT, then an LLM, then TTS. You get a transcript at every step, swappable layers and easier debugging.

That's the whole distinction you need for vendor selection. If you want the mechanics in depth, read how speech-to-speech voice agents work or our comparison of voice agent architectures. This post is about which vendor to pick.

Best speech-to-speech voice agent APIs in 2026

Prices as of October 28, 2026, taken from each vendor's own pricing or docs page. Accuracy figures come from AssemblyAI's published benchmarks, where the vendor is included.

Provider Architecture Accuracy evidence Pricing Control Languages
OpenAI GPT-Live-1 Native speech-to-speech Not in AssemblyAI's voice-agent benchmark $0.05 per minute, billed per second; backend model and tool usage charged separately Low: one vendor's model for all three layers Per OpenAI docs
OpenAI gpt-realtime-2.1 and 2.1-mini Native speech-to-speech OpenAI GPT Realtime: 11.78% mean WER on AssemblyAI's accented-English table (17th) 2.1: $32 per 1M audio input tokens, $64 per 1M audio output tokens. 2.1-mini: $10 and $20 Low: one vendor's model Per OpenAI docs
Google Gemini 3.8 Live Native audio model Not in AssemblyAI's voice-agent benchmark See Google's Gemini API pricing page Low: one vendor's model, tied to Google's stack 70
Amazon Nova 2 Sonic Native speech-to-speech (bidirectional streaming) Not in AssemblyAI's voice-agent benchmark See AWS pricing Low: one vendor's model; 8-minute connection limit English (4 locales), French, Italian, German, Spanish, Portuguese, Hindi
ElevenLabs ElevenAgents Managed cascade (STT, LLM, TTS, turn-taking model); optional realtime model Scribe v2 Realtime: 7.78% mean WER, 18.5% entity error See ElevenLabs pricing Medium: platform-configured components Its docs list 70+ in one section and 31 in the voice table
Deepgram Voice Agent API Managed cascade (configurable STT, LLM, TTS) Nova-3: 8.64% mean WER. Flux EN: 13.50% mean WER, 30.1% entity error Not listed on its Voice Agent docs page Medium: one WebSocket, vendor-hosted pipeline Depends on chosen models
AssemblyAI cascaded pipeline (Universal-3.6 Pro Realtime + any LLM + Text-to-Speech API) Cascaded pipeline you own Universal-3.6 Pro Realtime: 5.19% mean WER (1st of 22), 14.41% entity error (1st of 22) STT $0.45/hr, billed per second. TTS $15 per 1M characters. LLM at provider rates via LLM Gateway High: every layer is yours 32 for speech-to-text with code-switching; 6 for TTS

Accuracy rows refer to the AssemblyAI English voice-agent benchmark (mean of three noise conditions) unless noted. Benchmark transcripts aren't your transcripts, so test on your own calls.

The roundup, vendor by vendor

OpenAI GPT-Live-1 and gpt-realtime-2.1

The default speech-to-speech choice. OpenAI now has two pricing shapes. GPT-Live-1 bills $0.05 per minute, per second, and charges backend model and tool usage separately. gpt-realtime-2.1 bills per token, at $32 per 1M audio input tokens and $64 per 1M audio output tokens; gpt-realtime-2.1-mini is $10 and $20. gpt-realtime-mini is deprecated.

The trade-offs: per-call cost that's hard to forecast, and an input transcript that's secondary to the model. On AssemblyAI's accented-English table, OpenAI GPT Realtime posted 11.78% mean WER, 17th in the field. Best for speech-native experiences where tone matters more than entity accuracy. If you're already on it and shopping around, see our OpenAI Realtime API alternatives.

Google Gemini 3.8 Live

Gemini 3.8 Live is Google's current stable live model for voice agents, with an extended-thinking variant. It converses in 70 languages, takes 16 kHz PCM audio in and returns 24 kHz audio. Like OpenAI's models, one large model hears, reasons and speaks, so you can't swap the speech recognition layer. Best for multimodal agents already built on Google's stack.

Amazon Nova 2 Sonic

Nova 2 Sonic is a bidirectional streaming speech-to-speech model on Amazon Bedrock. It covers English (US, UK, India and Australia), French, Italian, German, Spanish, Portuguese and Hindi. Its docs set a connection limit of 8 minutes, so longer calls need the session-renewal pattern from AWS's samples. Best for teams standardized on AWS that can engineer around the connection cap.

ElevenLabs ElevenAgents

ElevenLabs renamed its agent product ElevenAgents. By its own description it coordinates four components: speech-to-text, an LLM, text-to-speech and a turn-taking model. So it's a managed cascade by default, with an optional realtime model. On AssemblyAI's benchmark, its Scribe v2 Realtime speech-to-text posted 7.78% mean WER and 18.5% entity error. Its docs give two different language counts (70+ in one section, 31 in the voice configuration table), so check your languages before committing.

Deepgram Voice Agent API

The Deepgram Voice Agent API is a WebSocket API that handles listening, thinking and speaking, with configurable STT, LLM and TTS. That makes it a managed cascade on Deepgram's infrastructure. On AssemblyAI's benchmark, Deepgram Nova-3 posted 8.64% mean WER and Deepgram Flux EN 13.50%, with 30.1% entity error for Flux EN. For a deeper look at platforms like this, plus Vapi and Telnyx, see our roundup of the best voice agent APIs.

Try Universal-3.6 Pro Realtime in the playground

Test it on your own recordings, with the names and numbers your agent has to get right.

Try playground

The cascaded alternative: Universal-3.6 Pro Realtime + LLM Gateway + Text-to-Speech API

AssemblyAI supplies both speech ends of a cascaded pipeline, plus a gateway to the LLM in the middle. Universal-3.6 Pro Realtime handles what the agent hears. AssemblyAI's Text-to-Speech API handles what it says. And LLM Gateway gives you one OpenAI-compatible endpoint for the reasoning step. Your orchestrator wires it together, whether that's LiveKit, Pipecat, Vapi or your own code.

Here's what each layer brings:

  • Accuracy first. Universal-3.6 Pro Realtime ranked 1st of 22 systems on AssemblyAI's English voice-agent benchmark, at 5.19% mean WER, and posted the lowest entity error rate, 14.41% against 18.19% for the next system.
  • Code-switching that holds up. It covers 32 languages with automatic detection, at 7.20% code-switching WER, 1st of 23.
  • Fast finals. On Pipecat's open STT benchmark, it delivered a 307 ms median time to final transcript.
  • Turn detection that isn't silence-only. It combines semantic context with voice activity, so a pause mid-account-number isn't treated as a finished turn.
  • Predictable speech pricing. $0.45/hr, billed per second, with no cap on open streams and new-session limits that scale with usage.
  • Any LLM, one string. LLM Gateway lives at https://llm-gateway.assemblyai.com/v1, uses your AssemblyAI API key, and exposes models like claude-sonnet-5-5, gpt-5.5, gemini-3.8-flash and deepseek-v4.1-flash. See the full list of available models.
  • Voice out, same vendor. AssemblyAI-TTS-1 is built for voice agents. It's trained on millions of hours of audio, including real conversational data, speaks six languages, and costs $15 per 1M characters. It delivers frontier-level quality and runs faster than other production voice agent TTS models. On the independent Cekura leaderboard, it ranks #1 on phone numbers and codes, and it ranks ahead of ElevenLabs v3 in the Artificial Analysis controlled-voice arena (as of November 12, 2026).
  • Regulated industries. AssemblyAI is considered a business associate under HIPAA and offers a Business Associate Addendum (BAA).

"We were searching for the best realtime ASR model for our voice agent pipeline in Fireflies. The new Universal 3.5 Pro speech model from Assembly is best so far in terms of accuracy, latency and language switching."

— Foysal Osmany, Software Engineer at Fireflies

Build the cascaded version. Start from the open-source reference agents for LiveKit or Pipecat. Point your LLM client at the LLM Gateway base URL, then add speech output with the Text-to-Speech quickstart. For a step-by-step build, follow our voice agent architecture tutorial, and for the production view see the voice AI stack for building agents.

Best fit: accuracy-critical phone agents, regulated industries and teams that don't want one model vendor owning every layer.

What changed in 2026

  • Universal-3.6 Pro Realtime became the streaming flagship. On AssemblyAI's English voice-agent benchmark it posted 5.19% WER, against 7.78% for ElevenLabs Scribe v2 Realtime, 8.64% for Deepgram Nova-3 and 13.50% for Deepgram Flux EN. Read the launch post.
  • OpenAI split its realtime pricing. GPT-Live-1 bills per minute plus model usage, while gpt-realtime-2.1 still bills per token.
  • Google moved to Gemini 3.8 Live. The earlier Flash Live preview models are now legacy.
  • "Speech-to-speech" platforms turned out to be cascades. ElevenAgents and the Deepgram Voice Agent API both run STT, LLM and TTS behind one API.
  • Speech in and speech out from one vendor. AssemblyAI's Text-to-Speech API means a cascaded pipeline no longer needs a separate voice vendor.

When speech-to-speech wins, and when cascaded wins

Pick a native speech-to-speech model when expressive output is the product, when the conversation is open-ended rather than transactional, and when you're comfortable with one vendor's model across all three layers.

Pick a cascaded pipeline when the agent must capture entities (account numbers, dates, addresses). Or when you need an audit trail of what was heard. Or when you want a specific LLM, or need data to stay in a region you choose.

The thing is, most voice agent failures start with a misheard word that the LLM then confidently acts on. A cascaded pipeline lets you see that word and fix the layer that failed.

How to choose in 5 steps

Run this loop on two or three shortlisted options, including at least one speech-to-speech model and one cascaded pipeline. It takes about a week.

  1. Build a 50-call test set from real traffic. Include accents, background noise, spelled-out names, alphanumerics (phone numbers, order IDs, emails, amounts) and interruptions. If you serve more than one language, add a non-English and code-switched slice of at least 10 calls. Our guide to multilingual voice agents covers that slice.
  2. Score entity accuracy, not just WER. Score the critical tokens separately: phone numbers, codes, names, dollar amounts. A provider that's 1 point better on WER but twice as bad on entities will fail your calls. For speech-to-speech models, you may have to infer what was heard from the reply, which is itself a finding.
  3. Time the response at p50 and p90. Measure from the moment the caller stops speaking to the first agent audio. Under about 800 ms at p50 feels natural, and slow p90 outliers are what callers remember. Check that the agent doesn't cut people off mid-thought.
  4. Test tool calls under real conditions. Wire three real tools. Measure round-trip time and error rate, and check whether the agent speaks during execution instead of going silent. Interrupt it mid-tool-call and confirm it recovers.
  5. Read every transcript, then run the security gate. Read what each option heard next to what it said. In parallel, send each vendor the procurement and security questionnaire checklist, covering SOC 2, BAA, residency, retention and concurrency.

Pick the option with the best entity accuracy whose p90 latency and security answers you can live with. Then load test it before launch.

Final words

Speech-to-speech models made voice agents feel natural. But the 2026 market tells a quieter story: even the "one API" agent products from ElevenLabs and Deepgram run a cascade underneath. The industry is converging on cascades for transactional work, and the real decision is who owns each layer.

If your agent mostly chats, a native model may be enough. If it books, verifies, pays or escalates, start from the speech layer that hears entities correctly, then choose the LLM and voice around it. Run the 5-step loop and let your own calls make the final call.

Build your cascaded voice agent

Stream audio to Universal-3.6 Pro Realtime with up to 333 hours of free streaming, then add any LLM and AssemblyAI text-to-speech.

Sign up free

Frequently Asked Questions (FAQs)

What is the best speech-to-speech API for building a voice agent in 2026?

OpenAI's GPT-Live-1 and gpt-realtime-2.1 are the default native speech-to-speech choices, with Gemini 3.8 Live and Amazon Nova 2 Sonic as the main alternatives inside Google and AWS. For agents that must capture names, numbers and codes, a cascaded pipeline built on Universal-3.6 Pro Realtime (5.19% WER and 14.41% entity error on AssemblyAI's benchmark), an LLM and a TTS model is usually the stronger choice.

Is speech-to-speech better than a cascaded STT-LLM-TTS pipeline?

Not for every job. Speech-to-speech models can be more expressive, but they give you less visibility into what was heard. Cascaded pipelines give you a transcript at every step, a choice of LLM and easier debugging, which matters most when the agent acts on entities.

Why not use speech-to-speech in production?

Many teams do, for open-ended conversation. Teams running transactional agents often move to a cascaded pipeline because they need an auditable transcript, a specific LLM, predictable per-layer pricing and control over entity accuracy. Per-session limits also matter: Nova 2 Sonic, for example, caps a connection at 8 minutes.

How do I build a speech-to-speech style voice agent with a cascaded pipeline?

Stream caller audio to Universal-3.6 Pro Realtime, send each finished turn to an LLM through LLM Gateway (https://llm-gateway.assemblyai.com/v1), and stream the reply through AssemblyAI's Text-to-Speech API. The LiveKit and Pipecat reference agents from AssemblyAI wire this up end to end, with turn detection and barge-in.

How much do speech-to-speech voice agent APIs cost?

As of October 28, 2026, OpenAI GPT-Live-1 costs $0.05 per minute plus backend model usage, and gpt-realtime-2.1 costs $32 per 1M audio input tokens and $64 per 1M audio output tokens. In a cascaded pipeline, Universal-3.6 Pro Realtime is $0.45/hr and AssemblyAI text-to-speech is $15 per 1M characters, with LLM usage at provider rates via LLM Gateway.

What happened to AssemblyAI's Voice Agent API?

AssemblyAI deprecated its Voice Agent API on October 26, 2026, and no longer offers it to new users. The recommended stack is Universal-3.6 Pro Realtime for streaming speech-to-text, LLM Gateway for the LLM and AssemblyAI's Text-to-Speech API for voice output, orchestrated with LiveKit, Pipecat or your own code.