In Miami, business calls move between English and Spanish inside a single thought. In Montreal, meetings shift between French and English depending on who's speaking and what the term of art is. In Mumbai and Indian diaspora cities everywhere, Hinglish blends English vocabulary into Hindi grammar so thoroughly that the two can't be separated at the sentence level.
A sentence might start in one language, borrow a phrase from another, and switch back inside a few seconds. Say this out loud and it sounds completely normal:
"I am traveling to Arizona and I want to comprar unos tiquets de avión."
Nothing about that is unusual to the person saying it. To a conventional transcription pipeline, it's a disaster.
Per-turn language detection is not code-switching support
This is the distinction that matters most, and it's the one most vendors blur.
The common architecture puts a language identification model in front of the recognizer. Audio arrives, the gateway classifies it as Spanish and routes it to the Spanish model. That works when someone speaks one language per turn. It fails the moment the language changes mid-utterance, because the routing decision was already made: the Spanish model gets English audio and transcribes it as the nearest Spanish-sounding guess.
It also costs you time. A detection gateway is a stage, and stages add latency to a path where you're counting milliseconds.
AssemblyAI-STT-3.7-Realtime has no language gateway. All 32 languages are handled in a single multilingual model, so a switch inside a clause is just more audio, with nothing to detect, re-route or restart.
| Approach | Handles mid-sentence switching | Added latency | Failure mode |
|---|---|---|---|
| Language detection gateway | No | A classification stage before recognition | Wrong-language transcription of the switched span |
| Per-turn language selection | Only between turns | Re-selection cost at each turn | Degrades on every switch inside a turn |
| Single unified multilingual model | Yes | No detection stage | Accuracy varies by language pair, not by switch point |
The 32 languages, and what "32" buys you
AssemblyAI-STT-3.7-Realtime and AssemblyAI-STT-3.7-Realtime-Agent stream these 32 languages with native code switching between them:
English · Spanish · French · German · Italian · Portuguese · Arabic · Danish · Dutch · Finnish · Hebrew · Hindi · Japanese · Mandarin · Norwegian · Swedish · Turkish · Vietnamese · Afrikaans · Cantonese · Catalan · Estonian · Galician · Korean · Marathi · Norwegian Nynorsk · Persian · Romanian · Russian · Urdu · Xhosa · Zulu
Language counts are among the most abused numbers in this industry, so read this one carefully. Thirty-two here doesn't mean 32 separate models you pick between. It means one model that hears all 32 at once, so any pair among them can alternate inside a sentence without you configuring anything. English–Spanish, French–English, Hindi–English and Arabic–French are all the same operation to the model.
Hinglish is the sharpest test case. English nouns and verbs sit inside Hindi sentence structure so densely that "which language is this?" isn't a well-formed question about a given clause. A router has to pick one and be wrong about half the words. A unified model doesn't have to pick.
For pre-recorded audio, AssemblyAI-STT-3.7 covers the same 32 languages and can fall back to Universal-2 for 99 languages in total when you add it to speech_models. That fallback is pre-recorded only; on streaming, 32 is 32. Our multilingual speech-to-text API guide covers the recorded-audio side.
Speak two languages in one sentence and watch the transcript keep up. Test streaming speed and accuracy on your own audio.
How accurate is real-time code switching?
On code-switched audio, AssemblyAI-STT-3.7-Realtime has a 6.67% word error rate and comes first on all five language pairs measured: English paired with Spanish, German, French, Portuguese and Italian. The figures are from the AssemblyAI-STT-3.7 launch post, with every vendor's realtime API at default settings, runs August to November 2026.
| Realtime model | Code-switched WER |
|---|---|
| AssemblyAI-STT-3.7-Realtime | 6.67% |
| ElevenLabs Scribe v3 Live | 8.32% |
| AWS Transcribe Streaming | 9.13% |
| Soniox Realtime v6 | 10.08% |
| Google Gemini 3.5 Transcribe Live | 10.16% |
| Meta Muse Voice Transcribe | 13.34% |
| Azure Realtime STT | 14.60% |
| Deepgram Flux Multi | 15.43% |
| Deepgram Nova-3 Multi | 17.13% |
The launch post's examples show what the lower number means in practice. On a test file with a Spanish sentence followed by an English one, the other providers shown either dropped the English sentence or wrote unrelated words in its place; AssemblyAI-STT-3.7-Realtime wrote both sentences.
Following switches is half the job; staying put is the other half. On non-English audio, words wrongly emitted in English fall to 0.033%, 44% below Soniox Realtime v6. The same model is the most accurate realtime API measured on Spanish, French, German and Galician.
Why entity accuracy is the number that matters here
Ask a vendor about multilingual accuracy and you'll get a word error rate. It's the wrong first question.
WER treats every word as equally important, so "um" and a customer's account number carry identical weight. In a multilingual conversation, the words that switch languages are rarely filler. They're product names, place names, technical terms and numbers: exactly the words a WER average smooths over, and exactly the words your application depends on. We made the longer argument in why word error rate is broken.
So look at entity accuracy. On AssemblyAI's benchmark of 12,460 English voice-agent calls, with no key terms or context for anyone, AssemblyAI-STT-3.7-Realtime-Agent has the lowest entity error rate of the 26 streaming models we ran, and fewer errors than the other providers shown on each entity class:
| Entity error rate (lower is better) | AssemblyAI-STT-3.7-Realtime-Agent | Soniox Realtime v6 | ElevenLabs Scribe v3 Live | Deepgram Nova-3 |
|---|---|---|---|---|
| All entities | 13.2% | 15.2% | 16.9% | 25.1% |
| Names | 10.18% | 12.47% | 14.02% | 23.69% |
| Phone numbers | 2.03% | 2.20% | 2.96% | 4.35% |
| Email addresses | 13.98% | 15.24% | 16.26% | 31.46% |
| Dates and times | 17.66% | 26.22% | 30.09% | 43.89% |
| Numbers and currency | 12.38% | 19.45% | 16.88% | 16.70% |
Accent is where multilingual speakers and entity errors meet. Callers who speak English as a second language are where every provider makes the most entity errors, and AssemblyAI-STT-3.7-Realtime-Agent has the lowest entity error rate in every one of the 12 speaker groups measured. The Realtime-Agent research post has the full methodology, and our piece on entity accuracy in speech-to-text explains why the metric matters.
Multilingual audio doesn't cost you latency either, since there's no detection stage to pay for. For voice agents, AssemblyAI-STT-3.7-Realtime-Agent returns the final transcript 86 ms after the caller stops speaking.
How to stream multilingual audio
Connect to the streaming WebSocket at wss://streaming.assemblyai.com/v3/ws and pass your connection parameters:
CONNECTION_PARAMS = {
"sample_rate": 16000,
"speech_model": "assemblyai-stt-3-7-realtime",
"language_codes": ["en", "es"],
}Four things are worth calling out:
speech_modelis singular on streaming. Pre-recorded transcription usesspeech_models, a plural list. Streaming takes one model name, and mixing them up is a common integration error.- Set the model explicitly. A config that omits it rides whatever the account default happens to be, which can change underneath you.
language_codessteers; it doesn't lock. Pass the languages you expect. The model still code-switches, but it's biased toward the languages you list. Omit the parameter for full multilingual behavior across all 32. You can update the list mid-session with anUpdateConfigurationmessage, and the change applies from the next turn. If you've fought with language steering before, our post on fixing language steering in streaming transcription covers the failure patterns.- Ask for the language when you need it. Set
language_detectiontotrueand each final turn reportslanguage_codeandlanguage_confidence. That's useful for routing a turn to the right translation target or support queue.
Model selection details are in the multilingual streaming docs, and our multilingual streaming build guide compares the two realtime models side by side.
Keyterms prompting, at no extra cost
Multilingual audio is where domain vocabulary breaks hardest, because a product name pronounced with Spanish phonology can land anywhere. Keyterms prompting fixes that: pass up to 100 terms, each 50 characters or less, as one JSON-encoded parameter, and swap them mid-stream with UpdateConfiguration as the conversation moves.
import json
CONNECTION_PARAMS = {
"sample_rate": 16000,
"speech_model": "assemblyai-stt-3-7-realtime",
"language_codes": ["en", "es"],
"keyterms_prompt": json.dumps(["Tarjeta Platino", "Plan Familiar", "AssemblyAI"]),
}Keyterms are included on both realtime models. A contextual prompt of up to 400 characters, describing the conversation in plain language, is a $0.05/hr add-on, and you can combine the two. The prompting and keyterms docs cover both.
Context is what makes code switching work
Some of the biggest gains on multilingual audio come from telling the model what the conversation is about.
Both realtime models carry context forward automatically. Earlier turns inform later ones, so a name established in minute one is still recognized in minute nine, even if the speaker switched languages in between. That matters for code switching, because the same entity often appears with different phonology in each language.
For voice agents, agent_context lets you pass the agent's most recent spoken reply, so the model hears the caller's answer through that lens. If the agent asked for an account number, a run of digits is read as digits rather than as similar-sounding words in whichever language the caller reaches for. You can update it each turn with UpdateConfiguration. Our write-up on conversation context for voice agents covers the mechanics.
Evaluate realtime speech-to-text with native code switching across 32 languages. Launch a pilot with clear docs and free credits.
What it costs
AssemblyAI-STT-3.7-Realtime is $0.30 per hour, and AssemblyAI-STT-3.7-Realtime-Agent is $0.45 per hour. Every language costs the same, with no premium for non-English transcription and no surcharge for code-switched audio.
One billing detail people miss: streaming is billed on session duration, meaning how long the WebSocket stays open, not on how much audio contains speech. Close idle sockets.
| Item | Price | Notes |
|---|---|---|
| AssemblyAI-STT-3.7-Realtime | $0.30/hr | All 32 languages, same rate |
| AssemblyAI-STT-3.7-Realtime-Agent | $0.45/hr | All 32 languages, same rate |
| Keyterms prompting | Included | Up to 100 terms, updatable mid-stream |
| Contextual prompting | +$0.05/hr | Combinable with keyterms |
| Speaker diarization | +$0.12/hr | Labels final at the end of each turn on AssemblyAI-STT-3.7-Realtime |
| Voice Focus | +$0.10/hr | AssemblyAI-STT-3.7-Realtime-Agent only |
New accounts get $50 in free credits for the speech APIs, enough to run a real pilot rather than a demo; LLM Gateway isn't included. Everything is pay-as-you-go with no upfront commitment. The full rate card lives on our pricing page.
Where this shows up in production
Voice agents serving multilingual markets. A Spanish-speaking customer switches to English for a technical term and back again. With a routing architecture, that's where the agent loses the thread. Our guide to building a multilingual voice agent walks through the full stack.
Realtime agent assist for multilingual support teams. Suggestions surfaced to a human agent are only useful if the transcript underneath them is right, and support conversations in bilingual markets code-switch constantly.
Meeting assistants across international offices. Teams that share a second language switch into it for speed and precision. AssemblyAI-STT-3.7-Realtime adds speaker labels that are final when each turn ends, so the notes say who said what in either language.
Clinical documentation. Patient consultations in multilingual communities often mix languages, with the clinician in one, the patient in another and sometimes an interpreter bridging both. Medical Mode improves accuracy on medication names in English, Spanish, German and French on the realtime models; it covers four languages, not 32. On compliance, AssemblyAI enables covered entities and their business associates subject to HIPAA to use the AssemblyAI services to process protected health information (PHI), and offers a standard Business Associate Addendum (BAA). See our medical transcription page for details.
Getting started
Set speech_model to assemblyai-stt-3-7-realtime, pass the language_codes you expect, open the socket and start streaming. You can also test it without writing anything: open the playground, pick streaming and speak two languages in one sentence.
The bigger lesson isn't about language counts. The industry spent years treating multilingual support as a routing problem: detect the language, dispatch to the right model, stitch the results. That framing produced systems that handled many languages well and mixed ones badly, which is backwards, because mixed is how people talk. A single model that recognizes every language in one pass drops the assumption that a span of audio has exactly one language, and that assumption was the bug all along.
Stream mixed-language audio through AssemblyAI-STT-3.7-Realtime and read the transcript live. Sign up and start with free credits.
Frequently Asked Questions (FAQs)
Can AssemblyAI handle multilingual audio with code-switching?
Yes. AssemblyAI-STT-3.7-Realtime handles mid-sentence code switching natively across 32 languages, including Hinglish, with no language detection gateway. On code-switched audio it has a 6.67% word error rate and comes first on all five language pairs measured, ahead of ElevenLabs Scribe v3 Live, AWS Transcribe Streaming, Soniox Realtime v6 and Google Gemini 3.5 Transcribe Live. For recorded audio, AssemblyAI-STT-3.7 has the lowest average code-switching error rate measured, 6.2%.
What languages does AssemblyAI's Voice AI support?
The realtime models support 32 languages: English, Spanish, French, German, Italian, Portuguese, Arabic, Danish, Dutch, Finnish, Hebrew, Hindi, Japanese, Mandarin, Norwegian, Swedish, Turkish, Vietnamese, Afrikaans, Cantonese, Catalan, Estonian, Galician, Korean, Marathi, Norwegian Nynorsk, Persian, Romanian, Russian, Urdu, Xhosa and Zulu. For recorded audio, AssemblyAI-STT-3.7 covers the same 32, and adding Universal-2 extends coverage to 99 languages.
How do transcription services handle multiple languages in the same audio?
Most services put a language identification model in front of the recognizer and route each turn to a language-specific model. That works for one language per turn and fails on mid-sentence switching, because the routing decision is already made. A unified multilingual model recognizes all its languages in one pass, so a switch inside a clause is just more audio. AssemblyAI's realtime models work this way and add no detection stage to the latency path.
Does AssemblyAI support real-time streaming transcription?
Yes. Streaming runs over a WebSocket at wss://streaming.assemblyai.com/v3/ws. AssemblyAI-STT-3.7-Realtime is $0.30/hr for live transcription, and AssemblyAI-STT-3.7-Realtime-Agent is $0.45/hr for voice agents, where it returns the final transcript 86 ms after the caller stops speaking. Both are billed on session duration. New accounts get $50 in free credits for the speech APIs; LLM Gateway isn't included.
What streaming features does AssemblyAI support?
Streaming includes 32-language code switching, language_codes steering, per-turn language reporting, keyterms prompting at no extra cost (up to 100 terms, updatable mid-stream), contextual prompting, automatic context carryover and agent_context for voice agents. Optional add-ons include realtime speaker labels on AssemblyAI-STT-3.7-Realtime, Voice Focus on AssemblyAI-STT-3.7-Realtime-Agent, Medical Mode and PII redaction.
AssemblyAI vs Speechmatics for multilingual transcription: how should I compare them?
Compare on your own code-switched audio and on entity accuracy, since word error rate and language counts hide the failures that break applications. On AssemblyAI's published realtime benchmarks, AssemblyAI-STT-3.7-Realtime-Agent is ahead of Speechmatics Enhanced Realtime on English voice-agent word error rate (4.53% vs 8.13%), and AssemblyAI-STT-3.7-Realtime is ahead on live speaker labels for telephone calls (10.46% vs 14.58% diarization error). Full tables are on the benchmarks page.