Now live New models for transcription and text-to-speech. AssemblyAI-STT-3.7 AssemblyAI-TTS-1
Voice agents · Multilingual

Multilingual voice agents

The people your voice agent serves speak dozens of languages — and many move between them mid-sentence. Reaching a global user base means understanding each caller in their own language, handling code-switching naturally, and holding accuracy on the names and numbers that matter in every one. Here's the full landscape across AssemblyAI's streaming models.

Language coverage

32 languages, native code-switching

32 languages at flagship accuracy with native mid-sentence code-switching
English Spanish French German Italian Portuguese Afrikaans Arabic Cantonese Catalan Danish Dutch Estonian Finnish Galician Hebrew Hindi Japanese Korean Mandarin Marathi Norwegian Bokmål Norwegian Nynorsk Persian Romanian Russian Swedish Turkish Urdu Vietnamese Xhosa Zulu

Only need a live transcript? AssemblyAI-STT-3.7-Realtime covers the same 32 languages at $0.30/hr.

The numbers that matter

Flagship languages

32

Flagship accuracy with native code-switching in a single stream on AssemblyAI-STT-3.7-Realtime-Agent.

Source: Multilingual voice agents

Code-switching

Native

Mid-sentence language switching — Hinglish, Spanglish, and more — without a declared primary language.

Source: Realtime STT

Cost-optimized

$0.30/hr

AssemblyAI-STT-3.7-Realtime covers the same 32 languages in one stream for live transcripts nobody has to answer.

Source: Realtime STT

Flagship streaming

$0.45/hr

AssemblyAI-STT-3.7-Realtime-Agent, all 32 languages, unlimited concurrent streams and no concurrency fees.

Source: Pricing

Customer story · Report 5

“We were searching for the best realtime ASR model for our voice agent pipeline in Fireflies. The new Universal-3.5 Pro speech model from Assembly is best so far in terms of accuracy, latency and language switching.”

— Foysal Osmany, Software Engineer at Fireflies

The challenge

The multilingual challenge — harder than it looks

Multilingual voice agents face compounding challenges: dialect variation within a language, code-switching mid-sentence, proper nouns that fit no single language model, and the need to keep accuracy in every language. Most providers force a tradeoff — a handful of languages with great accuracy, or a hundred with mediocre accuracy. The gap is false confidence: handling simple Spanish phrases in a demo doesn't mean handling a bilingual support call that switches between Spanish and English.

Recommended

AssemblyAI-STT-3.7-Realtime-Agent

32 languages, one stream

$0.45 /hr

Unlimited concurrent streams, no concurrency fees

  • Native mid-sentence code-switching
  • No primary language to declare
  • Keyterms prompting included
  • Works with LiveKit, Pipecat, Vapi, and Twilio SIP
Build a multilingual voice agent

Voice output

Text-to-Speech API

Six languages, ~90 ms to first audio

$15 /1M characters

Billed on input characters

  • English, Spanish, French, German, Italian, Portuguese
  • 4,000+ voices and instant voice cloning
  • Streaming WebSocket and HTTP APIs, word-level timestamps
  • US or EU processing at the same price
Learn more

Cost-optimized

AssemblyAI-STT-3.7-Realtime

32 languages, lower cost

$0.30 /hr

Live transcripts nobody has to answer: notetakers, captions, agent assist

  • Same 32 languages in one stream
  • Speaker labels, final at turn end
  • Upgrade to AssemblyAI-STT-3.7-Realtime-Agent when the agent talks back
  • Best for: live transcription on a budget
Learn more

Need a lower-cost stream? AssemblyAI-STT-3.7-Realtime runs the same 32 languages at $0.30/hr.

Code-switching

It never pauses to catch up

The model recognizes and preserves language switches mid-utterance, so bilingual calls flow naturally — no primary language to declare, and prompting handles the proper nouns and technical terms that span languages.

Live transcript

English → French

I said something like, “J'ai dit à mes étudiants que it's time to really pay attention to what the idea of code-switching is.”

English → Hindi

हम छह साल की उम्र में मिले थे, but I have an exam to give tomorrow.

English → Mandarin

But this sentence, 我让你干工作了 — you can see the tone at the end tells you the situation is different from before.

Architecture patterns

Architecture patterns

Three ways to structure a multilingual deployment, from a single-market launch to premium multi-market accuracy to a cost-optimized core-market build — all on the same AssemblyAI foundation.

Single-market launch

Start from the LiveKit or Pipecat reference implementation, which wires AssemblyAI-STT-3.7-Realtime-Agent, LLM Gateway, and the Text-to-Speech API together on one API key and one bill. Launch in one of the six Text-to-Speech languages, expand later.

LiveKit or Pipecat

Multi-market, premium accuracy

AssemblyAI-STT-3.7-Realtime-Agent across 32 languages, with LLM Gateway or your own LLM. Use the Text-to-Speech API in its six languages, and pair AssemblyAI-STT-3.7-Realtime-Agent with another TTS provider for the rest.

AssemblyAI-STT-3.7-Realtime-Agent

Cost-optimized core markets

AssemblyAI-STT-3.7-Realtime for live transcripts across the same 32 languages at lower cost. Upgrade to AssemblyAI-STT-3.7-Realtime-Agent when the agent has to answer the caller.

AssemblyAI-STT-3.7-Realtime

Ship voice agents for a global user base

Get your API key and build in your primary markets today — scale across all 32 languages without switching providers.

Frequently asked questions

What is the best speech-to-text API for multiple languages?

AssemblyAI supports 32 languages at flagship accuracy on AssemblyAI-STT-3.7-Realtime-Agent, with native mid-sentence code-switching in a single stream. For live transcripts nobody has to answer, AssemblyAI-STT-3.7-Realtime covers the same 32 languages at $0.30/hr. Premium accuracy plus a cost-optimized option on one account is what sets it apart for multilingual voice agents.

How do I build a multilingual voice agent?

Start from the LiveKit or Pipecat reference implementation, which wires AssemblyAI-STT-3.7-Realtime-Agent, LLM Gateway, and the Text-to-Speech API together on one API key and one bill. AssemblyAI-STT-3.7-Realtime-Agent handles code-switching across 32 languages with no primary language to declare, LLM Gateway routes the transcript to the LLM of your choice (or bring your own), and your orchestrator handles tool calls and reconnects. The Text-to-Speech API covers English, Spanish, French, German, Italian, and Portuguese; for other languages, pair AssemblyAI-STT-3.7-Realtime-Agent with another TTS provider.

Which languages does AssemblyAI support for voice agents?

32 at flagship accuracy on AssemblyAI-STT-3.7-Realtime-Agent: Afrikaans, Arabic, Cantonese, Catalan, Danish, Dutch, English, Estonian, Finnish, French, Galician, German, Hebrew, Hindi, Italian, Japanese, Korean, Mandarin, Marathi, Norwegian (Bokmål and Nynorsk), Persian, Portuguese, Romanian, Russian, Spanish, Swedish, Turkish, Urdu, Vietnamese, Xhosa, and Zulu — with native code-switching. For live transcripts nobody has to answer, AssemblyAI-STT-3.7-Realtime covers the same 32 languages in one stream at $0.30/hr. The Text-to-Speech API speaks English, Spanish, French, German, Italian, and Portuguese. For non-real-time use like post-call analytics, pre-recorded transcription adds automatic language detection across 99+ languages.

Can AssemblyAI handle code-switching between languages?

Yes — natively. AssemblyAI-STT-3.7-Realtime-Agent recognizes and preserves language switches mid-utterance (Hinglish, Spanglish, and more) in a single stream, without requiring you to declare a primary language. Keyterms prompting handles the proper nouns and technical terms that span languages.

How does language detection work for a multilingual voice agent?

For live streaming you don't declare a language — AssemblyAI-STT-3.7-Realtime-Agent handles code-switching across its 32 languages automatically, and you select the model with the speech_model parameter (for example, assemblyai-stt-3-7-realtime-agent, or assemblyai-stt-3-7-realtime for the cost-optimized tier). For pre-recorded audio, automatic language detection identifies the dominant language and routes to the best model across 99+ languages.

How much does multilingual transcription cost?

There's no language surcharge on the flagship models. AssemblyAI-STT-3.7-Realtime-Agent is $0.45/hr across all 32 languages, with unlimited concurrent streams and no concurrency fees. The Text-to-Speech API is $15 per 1M characters, billed on input characters, at the same price in the US or EU and with no spend commitment to unlock concurrency. AssemblyAI-STT-3.7-Realtime is the lower-cost option at $0.30/hr for live transcripts nobody has to answer, across the same 32 languages.