Now live New models for transcription and text-to-speech. AssemblyAI-STT-3.7 AssemblyAI-TTS-1
Insights & Use Cases

Real-time Spanish to English speech translation with an API (Python)

Translate Spanish speech to English in real time: stream audio to STT-3.7-Realtime, get English per turn on the same WebSocket, and optionally voice it.

Abstract green sphere illustration

Written by

Kelsey Foster

Published on

18 November 2026

To translate a Spanish recording once, a consumer app is fine. To translate Spanish speech to English inside your own product, like a call-routing tool, a radio monitor, a bilingual help desk or a live event, you need an API. This guide builds the API version: stream Spanish audio to AssemblyAI, get the English translation of every turn back on the same WebSocket, and optionally hear the English spoken aloud. A short section at the end covers recorded Spanish files.

Apps vs an API for Spanish to English speech translation

The search results for this topic are mostly apps: paste a file or hold up your phone, read the English. They work for a one-off.

An API is for when the translation has to live inside software you ship. You control where the audio comes from, how the English is shown or stored, and what happens next, like routing a Spanish call, archiving an English transcript or reading the translation aloud. The rest of this post is the developer path, in Python.

How the pipeline works for ES→EN

This is a cascaded pipeline: speech-to-text, then text translation, then optional text to speech. The difference from rolling your own is that the first two steps share one connection.

  1. You open one streaming session on AssemblyAI-STT-3.7-Realtime with an llm_gateway object that holds your Spanish-to-English prompt.
  2. You stream Spanish audio. Turn messages come back in Spanish as people speak.
  3. When a turn ends, the server sends the final Spanish transcript to LLM Gateway with your prompt and returns the English as an LLMGatewayResponse message with the same turn_order (real-time translation docs).
  4. Optionally, you pass each English line to text to speech.

For why cascaded pipelines are the default today, and where end-to-end speech translation fits, see the real-time speech translation pillar post. This guide sticks to what's specific to Spanish.

AssemblyAI-STT-3.7-Realtime is $0.30/hr and supports 32 languages, Spanish among them (pricing). It fits one-way translation, where someone reads or hears the English. If your software answers the Spanish speaker, like a bilingual voice agent, use AssemblyAI-STT-3.7-Realtime-Agent ($0.45/hr). The translation setup is identical.

Translate Spanish Speech in Real Time

Stream Spanish audio to AssemblyAI-STT-3.7-Realtime and get English back per turn on the same WebSocket. Create a free account to get your API key.

Sign up free

Spanish-specific settings

Tell the model which languages to expect

Streaming is multilingual by default and code-switches across all 32 supported languages with no configuration. When you know the speakers are Spanish, pass language_codes to steer the model toward Spanish (multilingual transcription docs):

  • ["es"] for Spanish speakers.
  • ["es", "en"] for speakers who mix Spanish and English. The model still code-switches, with a strong bias toward the two languages you list.

You can also change language_codes mid-session with an UpdateConfiguration message, which takes effect from the next turn.

Write the prompt for the Spanish your users speak

Spanish isn't one vocabulary. "Ahorita" means different things in Mexico and the Caribbean, "vale" is everywhere in Spain, and "guagua" is a bus in Cuba and a baby in Chile. Numbers are written differently too: "1.500,00 €" uses a period for thousands and a comma for decimals. The prompt from the companion repo handles these explicitly:

ES_TO_EN_PROMPT = """Translate the current turn from Spanish into natural American English.
Reply with only the English translation, no notes or quotation marks.

Rules:
- Speakers may be from Mexico, Spain, the Caribbean, Central or South America. Translate regional words by meaning (for example "ahorita", "vale", "guagua", "chévere"), not word for word.
- If the speaker switches into English mid-sentence, keep their English as spoken.
- Keep names of people, places, companies and products as spoken.
- Write numbers, dates and money the US way: "1.500,00 €" becomes "€1,500.00", "el 3 de mayo" becomes "May 3", "las 19:30" becomes "7:30 PM".
- Use the earlier turns only for context. Do not translate them again.

Earlier turns: {{history_3}}

Current turn: {{turn}}"""

Two template variables do the work. The server replaces {{turn}} with the final Spanish transcript of each turn, and {{history_3}} with up to three earlier turns whose translations have already completed. History is the main lever for translation quality: "Sí, ese." is ambiguous alone and clear after the question it answers. Change the target from American to British English, or swap the date format, by editing the rules.

Build it: stream Spanish audio and print English

The companion repo, spanish-to-english-speech-translation, uses the assemblyai Python SDK 1.10.1.

pip install assemblyai==1.10.1
export ASSEMBLYAI_API_KEY=your_api_key_here

Connection parameters

from assemblyai.streaming.v3 import RealTimeParameters
from assemblyai.streaming.v3.models import LLMGatewayConfig, LLMGatewayMessage

from prompts import ES_TO_EN_PROMPT


def make_params(spanglish=False):
    return RealTimeParameters(
        sample_rate=16000,
        speech_model="assemblyai-stt-3-7-realtime",  # AssemblyAI-STT-3.7-Realtime
        language_codes=["es", "en"] if spanglish else ["es"],
        llm_gateway=LLMGatewayConfig(
            model="qwen3.5-4b-32k-fast",
            messages=[LLMGatewayMessage(role="user", content=ES_TO_EN_PROMPT)],
            max_tokens=400,
        ),
    )

qwen3.5-4b-32k-fast is the small, fast model the docs use for translation. If you leave model out, the gateway defaults to gemini-2.5-flash-lite. A few hundred max_tokens covers any single turn.

Pair each English line with its Spanish turn

Translations can arrive out of order when turns are short and close together, and if a gateway call fails or times out, no LLMGatewayResponse is sent for that turn. So the handler stores Spanish finals by turn_order, prints a pair when the English arrives, and prints a placeholder if nothing comes back within three seconds:

import threading


class PairPrinter:
    def __init__(self, speaker=None, fallback_after=3.0):
        self.spanish, self.done = {}, set()
        self.speaker, self.fallback_after = speaker, fallback_after
        self._lock = threading.Lock()

    def on_turn(self, client, event):
        if not event.end_of_turn or not event.transcript:
            return  # partials stay in Spanish and aren't translated
        with self._lock:
            self.spanish[event.turn_order] = event.transcript
        timer = threading.Timer(self.fallback_after, self._no_translation,
                                args=(event.turn_order,))
        timer.daemon = True
        timer.start()

    def on_llm_response(self, client, event):
        try:
            english = event.data["choices"][0]["message"]["content"].strip()
        except (KeyError, IndexError, TypeError, AttributeError):
            english = ""
        with self._lock:
            if event.turn_order in self.done or not english:
                return
            self.done.add(event.turn_order)
            spanish = self.spanish.get(event.turn_order, event.transcript)
        print(f"[{event.turn_order}] ES: {spanish}\n    EN: {english}", flush=True)
        if self.speaker is not None:
            self.speaker.say(english)

    def _no_translation(self, turn_order):
        with self._lock:
            if turn_order in self.done:
                return
            self.done.add(turn_order)
            spanish = self.spanish[turn_order]
        print(f"[{turn_order}] ES: {spanish}\n    EN: (no translation)", flush=True)

Stream a file, a URL or a device

audio_source.py in the repo yields 50 ms chunks of 16 kHz, 16-bit mono PCM, paced to real time. A 16 kHz mono WAV needs nothing else. Any other file, or an internet radio or HLS stream URL, goes through ffmpeg (-re keeps it at real-time speed). Then wire it together:

import os

from assemblyai.streaming.v3 import (
    RealTimeEvents,
    RealTimeTranscriber,
    RealTimeTranscriberOptions,
)

from audio_source import open_source

printer = PairPrinter()
client = RealTimeTranscriber(
    RealTimeTranscriberOptions(api_host="streaming.assemblyai.com"),
    api_key=os.environ["ASSEMBLYAI_API_KEY"],
)
client.on(RealTimeEvents.Turn, printer.on_turn)
client.on(RealTimeEvents.LLMGatewayResponse, printer.on_llm_response)
client.on(RealTimeEvents.Error, lambda c, err: print(f"Streaming error: {err}"))

client.connect(make_params(spanglish=False))
try:
    client.stream(open_source("entrevista.wav"))
finally:
    client.disconnect(terminate=True)

Run it with python stream_translate.py entrevista.wav, or point it at a stream with python stream_translate.py https://example.com/radio/stream.mp3 --spanglish. Output looks like this:

[0] ES: Buenos días, ¿cómo está?
    EN: Good morning, how are you?
[1] ES: Son 1.500,00 € en total.
    EN: That's €1,500.00 in total.

The docs say that with a small model such as qwen3.5-4b-32k-fast, the translation typically lands within about 100 ms of the final turn, and it never delays the Spanish Turn message itself.

Optional: hear the English with text to speech

For a listener who'd rather hear than read, send each English line to AssemblyAI text to speech. English is one of the model's six languages, and aai-kyla is the default English voice in the TTS docs.

Open one TTS session for the whole run and speak every line through it. Opening a session per sentence pays the connection handshake each time and counts against the limit on new sessions per minute (TTS quickstart). A session closes after two minutes with no frames in either direction, so a quiet stretch of Spanish audio needs a KeepAlive:

import queue
import threading

import assemblyai as aai


class EnglishSpeaker:
    def __init__(self, api_key, out_path="english.pcm"):
        self._config = aai.TTSConfig(voice_id="aai-kyla", language="english")
        self._api_key = api_key
        self._queue = queue.Queue()
        self._out = open(out_path, "wb")  # 24 kHz 16-bit mono PCM
        self._thread = threading.Thread(target=self._run, daemon=True)
        self._thread.start()

    def say(self, text):
        if text.strip():
            self._queue.put(text)

    def close(self):
        self._queue.put(None)
        self._thread.join(timeout=30)
        self._out.close()

    def _run(self):
        with aai.TTSSession(self._config, api_key=self._api_key) as session:
            while True:
                try:
                    text = self._queue.get(timeout=60)
                except queue.Empty:
                    session.keep_alive()  # restart the two-minute idle timer
                    continue
                if text is None:
                    return
                for chunk in session.speak(text):
                    self._out.write(chunk)

The repo version also pipes audio to ffplay when it's installed, so python stream_translate.py entrevista.wav --speak plays English as the Spanish comes in. Text to speech is billed on input characters at $15 per 1M characters. For picking a different voice, see how to choose a voice for an AI voice agent.

Hear Spanish Become English

Try streaming transcription and text to speech in the playground, then run the companion repo on your own Spanish audio.

Try playground

Recorded Spanish audio: translate the whole file

If the Spanish audio is already recorded, like an interview, a voicemail archive or a podcast, you don't need streaming. Pre-recorded transcription with the Speech Understanding translation feature returns the Spanish transcript and the English translation from one request:

import assemblyai as aai

aai.settings.api_key = "<YOUR_API_KEY>"

config = aai.TranscriptionConfig(
    speech_models=["universal-3-5-pro", "universal-2"],
    language_detection=True,
    speaker_labels=True,
    speech_understanding=aai.SpeechUnderstandingRequest(
        request=aai.SpeechUnderstandingFeatureRequests(
            translation=aai.TranslationRequest(target_languages=["en"], formal=False)
        )
    ),
)

transcript = aai.Transcriber().transcribe("entrevista.mp3", config)
print(transcript.text)                                  # Spanish
print(transcript.json_response["translated_texts"]["en"])  # English

Translation adds $0.06/hr on top of transcription and can target any of 86 languages (translation docs). With speaker labels on, you also get per-speaker translated utterances. For a deeper batch build that translates into several languages and re-voices the result, see build an AI voice translator in Python.

Real time (streaming + llm_gateway) Recorded (pre-recorded + Speech Understanding)
Input Live audio, or a file replayed at real-time pace A file path or URL
When English arrives Per turn, about 100 ms after the turn ends (small model, per the docs) With the finished transcript
Model AssemblyAI-STT-3.7-Realtime Universal-3.5 Pro
Pricing unit $0.30/hr of session time, plus LLM Gateway tokens Transcription per hour of audio, plus $0.06/hr for translation
Translation control Your own prompt: regional rules, formats, context Target languages and formal or informal style

Accuracy tips and testing

Translation quality depends on the Spanish transcript first and the prompt second, so test both.

  • Test with dialects your users speak. A Mexico City customer line and a Madrid sports broadcast stress different vocabulary. Collect a few minutes of each and read the pairs side by side.
  • Test code-switching on purpose. Record speakers who move between Spanish and English mid-sentence, and compare ["es"] with ["es", "en"].
  • Check the entities. Names, prices, dates and phone numbers are where a translation goes visibly wrong. Grep the English for currency symbols and digits and compare them with the Spanish.
  • Keep history on for conversations. Turn off or shorten history for monologues with independent sentences, since every history turn adds prompt tokens.
  • Watch the fallbacks. If you see many "(no translation)" lines, check your gateway errors, and try a slightly larger model from the LLM Gateway model list.

For Spanish transcription features beyond translation, see the Spanish language page.

Wrapping up

Spanish to English speech translation with an API comes down to one streaming session with language_codes and an llm_gateway prompt written for real Spanish, a handler that pairs translations by turn_order, and an optional TTS session for spoken English. Recorded files take one pre-recorded request instead.

To translate a multi-speaker meeting into several languages at once with speaker labels, see our guide to translated captions for meetings. For a vendor comparison, read best real-time speech translation APIs.

Build Spanish to English Translation Today

Start free with AssemblyAI-STT-3.7-Realtime and llm_gateway translation, and add spoken English with text to speech when you need it.

Sign up free

Frequently Asked Questions (FAQs)

How do I translate Spanish audio to English with an API?

For live audio, open a streaming session on AssemblyAI-STT-3.7-Realtime with language_codes: ["es"] and an llm_gateway prompt that translates Spanish into English. Each final Spanish turn comes back as a Turn message, followed by its English translation as an LLMGatewayResponse with the same turn_order. For a recorded file, send one pre-recorded transcription request with Speech Understanding translation and target_languages: ["en"].

Can it handle speakers who mix Spanish and English?

Yes. Streaming code-switches natively, and setting language_codes to ["es", "en"] biases the model toward those two languages while still following switches mid-sentence. Tell the translation prompt to keep any English the speaker already used.

Does it work for Mexican and Spain Spanish?

Both are Spanish to the speech model, which supports Spanish among its 32 streaming languages. Regional vocabulary is handled in the translation prompt: name the regions you expect and ask for meaning-based translation of words like "ahorita" or "vale". Test with audio from the regions your users come from.

How do I translate a recorded Spanish file instead of live audio?

Use pre-recorded transcription with the Speech Understanding translation feature and target_languages: ["en"]. The response includes the Spanish transcript and the English text under translated_texts, and per-speaker translated utterances if speaker labels are on. Translation adds $0.06 per hour of audio.

Can the English translation be spoken aloud?

Yes. Send each English line to AssemblyAI text to speech with the aai-kyla voice and language="english", over one long-lived session. Send a KeepAlive during quiet stretches, since a session closes after two minutes with no frames.

How is real-time Spanish to English translation billed?

Streaming on AssemblyAI-STT-3.7-Realtime is $0.30 per hour of session time, and the translation is billed on LLM Gateway tokens for the model you pick. Text to speech, if you add it, is $15 per 1M input characters.