Real-time transcription in Python with Universal-3.6 Pro Realtime: Build in 5 minutes

Real-time transcription turns speech into text while someone is still talking. Your Python app streams small chunks of audio over a WebSocket and gets text back as turns form, instead of uploading a finished file. In this tutorial you’ll build a working real-time transcriber with Universal-3.6 Pro Realtime in about five minutes. The first version needs no microphone.

Here’s what makes real-time transcription different from batch transcription:

• WebSocket connection. An open channel between your app and the service, instead of a one-off file upload.
• Streaming session. Audio is processed continuously until you close the connection.
• Turn events. Each event carries the transcript of one speaking turn, plus timing and an end_of_turn flag.
• Turn detection. The model decides when a speaker has finished, using what was said, not just silence.

When to use streaming transcription

Use streaming when your application has to react to speech as it happens. Voice agents can’t wait two seconds for a transcript without breaking the conversation. Live captions have to stay in sync with the speaker. Meeting tools that show notes as people talk need text immediately.

If you’re transcribing recordings such as podcasts, lectures, or call archives, use pre-recorded transcription instead. It’s cheaper at $0.21/hr on Universal-3.5 Pro, and speed doesn’t matter after the fact.

Before you start

• Python 3.8 or later.
• An AssemblyAI API key. Sign up free to get one.
• A microphone, only for the optional mic example later.

Universal-3.6 Pro Realtime costs $0.45/hr base, billed on session duration: the time your WebSocket stays open, not the amount of audio you send. Close sessions when you’re done. Free accounts can open 5 new streaming sessions per minute. Paid accounts start at 100 new sessions per minute, and that limit scales up automatically by 10% each minute you use 70% or more of it. There’s no cap on how many sessions you keep open. Details are in the streaming rate limits docs.

Set up your Python environment

mkdir realtime-transcription
cd realtime-transcription
python -m venv venv
source venv/bin/activate        # Windows: .\venv\Scripts\activate
pip install assemblyai requests

Set your API key as an environment variable so it never lands in your code:

export ASSEMBLYAI_API_KEY=your_api_key_here

Stream your first session (no microphone needed)

This script streams a public internet radio feed for 25 seconds and prints each turn. Because it doesn’t touch a microphone, it runs anywhere: your laptop, a server, or a coding agent’s sandbox. Save it as transcribe.py:

import os
import time

import requests
from assemblyai.streaming.v3 import (
    BeginEvent,
    Encoding,
    RealTimeError,
    RealTimeEvents,
    RealTimeParameters,
    RealTimeTranscriber,
    RealTimeTranscriberOptions,
    TerminationEvent,
    TurnEvent,
)

# A live AAC internet radio stream, so no microphone is needed.
STREAM_URL = "https://14123.live.streamtheworld.com/WBBRAMAAC.aac"
RUN_SECONDS = 25


def on_begin(client: RealTimeTranscriber, event: BeginEvent):
    print(f"Session started: {event.id}")


def on_turn(client: RealTimeTranscriber, event: TurnEvent):
    if event.end_of_turn:
        print(f"FINAL:   {event.transcript}")
    else:
        print(f"partial: {event.transcript}")


def on_terminated(client: RealTimeTranscriber, event: TerminationEvent):
    print(f"Session ended: {event.audio_duration_seconds}s of audio processed")


def on_error(client: RealTimeTranscriber, error: RealTimeError):
    print(f"Error: {error}")


def main():
    client = RealTimeTranscriber(
        RealTimeTranscriberOptions(terminate_timeout=30.0),
        api_key=os.environ["ASSEMBLYAI_API_KEY"],
    )

    client.on(RealTimeEvents.Begin, on_begin)
    client.on(RealTimeEvents.Turn, on_turn)
    client.on(RealTimeEvents.Termination, on_terminated)
    client.on(RealTimeEvents.Error, on_error)

    # AAC carries its own sample rate, so no sample_rate is needed here.
    client.connect(
        RealTimeParameters(
            speech_model="universal-3-6-pro",
            encoding=Encoding.aac,
            mode="balanced",
        )
    )

    response = requests.get(STREAM_URL, stream=True)
    deadline = time.time() + RUN_SECONDS
    try:
        for chunk in response.iter_content(chunk_size=4096):
            client.stream(chunk)
            if time.time() > deadline:
                break
    finally:
        response.close()
        # Terminate finalizes the open turn and stops billing for the session.
        client.disconnect(terminate=True)


if __name__ == "__main__":
    main()

Run it with python transcribe.py. You’ll see partials update as the announcer talks, then a final line at each natural pause.

A few details matter here:

• speech_model="universal-3-6-pro" selects Universal-3.6 Pro Realtime. It’s also the default if you leave speech_model out.
• mode="balanced" is the default trade-off between speed and accuracy. More on modes below.
• client.disconnect(terminate=True) in the finally block closes the session cleanly. Sessions you don’t terminate auto-close after 3 hours and are billed for the full time.

Understand turn events

Universal-3.6 Pro Realtime returns transcripts as turns. A turn is one continuous stretch of speech from one speaker.

Two rules make streaming code simple:

• Replace, don’t append. Every turn event re-transcribes the whole turn, so overwrite the previous partial with each new one.
• Act on finals. A final (end_of_turn: true) is the most accurate version and never changes. Send finals to your LLM or database. Use partials for live captions and barge-in.

Finals are always formatted on Universal-3.6 Pro Realtime. There’s no formatting flag to set.

Stream from your microphone

Once the file version works, swap the audio source for your microphone. Install PyAudio and the PortAudio library for your OS first:

pip install pyaudio

Save this as mic.py. It reuses the handlers from transcribe.py:

import os

import pyaudio
from assemblyai.streaming.v3 import (
    RealTimeEvents,
    RealTimeParameters,
    RealTimeTranscriber,
    RealTimeTranscriberOptions,
)

from transcribe import on_begin, on_turn, on_terminated, on_error

SAMPLE_RATE = 16000
FRAMES_PER_BUFFER = 800  # 50 ms of audio at 16 kHz


def main():
    client = RealTimeTranscriber(
        RealTimeTranscriberOptions(),
        api_key=os.environ["ASSEMBLYAI_API_KEY"],
    )
    client.on(RealTimeEvents.Begin, on_begin)
    client.on(RealTimeEvents.Turn, on_turn)
    client.on(RealTimeEvents.Termination, on_terminated)
    client.on(RealTimeEvents.Error, on_error)

    client.connect(
        RealTimeParameters(
            speech_model="universal-3-6-pro",
            sample_rate=SAMPLE_RATE,
            mode="balanced",
        )
    )

    pa = pyaudio.PyAudio()
    mic = pa.open(
        format=pyaudio.paInt16,
        channels=1,
        rate=SAMPLE_RATE,
        input=True,
        frames_per_buffer=FRAMES_PER_BUFFER,
    )
    print("Speak into your microphone. Press Ctrl+C to stop.")
    try:
        while True:
            client.stream(mic.read(FRAMES_PER_BUFFER, exception_on_overflow=False))
    except KeyboardInterrupt:
        pass
    finally:
        mic.stop_stream()
        mic.close()
        pa.terminate()
        client.disconnect(terminate=True)


if __name__ == "__main__":
    main()

Use 16 kHz mono 16-bit PCM for microphone audio. Higher sample rates add bandwidth without improving accuracy.

Tune turn detection

Universal-3.6 Pro Realtime doesn’t end turns on silence alone. When a speaker pauses for min_turn_silence, the model checks whether what’s been said reads as complete. If it does, the turn ends. If not, you get a partial and the turn stays open until max_turn_silence forces it closed.

The easiest way to tune this is the mode preset, which sets both values for you:

Pass explicit values only when a preset doesn’t fit. A common case is entity capture. Callers pause between the groups of a phone number, and each pause can read as a finished thought. When your agent asks for a number, raise min_turn_silence, then set it back after the answer. The SDK’s set_params can’t change mode, so restore the preset’s value directly (128 ms for balanced). Over a raw WebSocket, you can re-send {"type": "UpdateConfiguration", "mode": "balanced"} instead:

from assemblyai.streaming.v3 import RealTimeSessionParameters

# Before asking for a phone number or card number
client.set_params(RealTimeSessionParameters(min_turn_silence=1000))

# After the caller answers, restore the balanced preset's 128 ms
client.set_params(RealTimeSessionParameters(min_turn_silence=128))

If you run your own turn detection, end the current turn immediately with:

client.force_endpoint()

The turn detection docs cover the full turn lifecycle, including the SpeechStarted event you can use as a barge-in signal.

Update keyterms mid-stream

Keyterms prompting is included with Universal-3.6 Pro Realtime. You can swap the list during a session without reconnecting, which suits voice agents that move through conversation stages:

from assemblyai.streaming.v3 import RealTimeSessionParameters

STAGE_KEYTERMS = {
    "identification": ["account number", "member ID", "date of birth"],
    "billing": ["autopay", "statement balance", "billing address"],
    "support": ["router", "firmware update", "error code"],
}


def enter_stage(client, stage):
    client.set_params(RealTimeSessionParameters(keyterms_prompt=STAGE_KEYTERMS.get(stage, [])))

Keep each list short and specific to the stage. Long lists can hurt words the model already gets right. We covered the failure modes in when keyterm prompting backfires, and the upside in keyterm prompting for real-time accuracy.

Use temporary tokens in client apps

Never ship your API key in browser or mobile code. Generate a short-lived token on your server and hand it to the client:

# server.py
import os

from flask import Flask, jsonify
from assemblyai.streaming.v3 import RealTimeTranscriber

app = Flask(__name__)


@app.route("/token")
def token():
    client = RealTimeTranscriber(api_key=os.environ["ASSEMBLYAI_API_KEY"])
    return jsonify(
        {
            "token": client.create_temporary_token(
                expires_in_seconds=600,
                max_session_duration_seconds=1800,
            )
        }
    )

# client side
import requests
from assemblyai.streaming.v3 import (
    RealTimeParameters,
    RealTimeTranscriber,
    RealTimeTranscriberOptions,
)

token = requests.get("https://your-server.example.com/token").json()["token"]
client = RealTimeTranscriber(RealTimeTranscriberOptions(token=token))
client.connect(RealTimeParameters(speech_model="universal-3-6-pro", sample_rate=16000))

expires_in_seconds can be 1 to 600. A token stays valid until it expires and can start more than one session, so request a fresh token per session if you want tight control.

A note on older code samples

Python SDK 1.0 (August 2026) renamed the streaming classes. If you find tutorials using StreamingClient, StreamingParameters, and StreamingEvents, that’s the earlier naming. They map one-to-one to RealTimeTranscriber, RealTimeParameters, and RealTimeEvents. Use the new names for anything you write today.

Real-time transcription use cases

• Voice agents. Customer service, scheduling, and intake agents that need accurate names, numbers, and addresses. See our Python voice agent tutorial.
• Live captions. Meetings, lectures, and events, with partials for speed and finals for the record.
• Meeting notes. Running transcripts with speaker labels. See streaming speaker diarization.
• Clinical documentation. Real-time scribes that need medical vocabulary right.
• Multilingual apps. Universal-3.6 Pro Realtime code-switches natively across 32 languages.

How fast is it?

On Pipecat’s open STT benchmark, Universal-3.6 Pro Realtime posts 0.96% pooled semantic WER with a 307 ms median time to first speech. Median endpoint latency is 537 ms (September 29, 2026). On AssemblyAI’s English voice-agent benchmark of 12,460 scenarios, mean WER is 5.19%. That’s the number that matters most for this tutorial: when your code acts on a final, the words are right.

Python makes the plumbing short. The model is what makes it usable. Start with the no-mic script, add your microphone, and tune mode before touching individual silence values. For a broader tour of Python options, see Python speech recognition in 2026.

Frequently asked questions

How do I do real-time speech-to-text in Python?

Install the AssemblyAI SDK with pip install assemblyai, create a RealTimeTranscriber, register a handler for turn events, connect with speech_model="universal-3-6-pro", and stream audio chunks with client.stream(). Call client.disconnect(terminate=True) when you’re done.

What’s the difference between StreamingClient and RealTimeTranscriber?

They’re the same streaming client under two names. Python SDK 1.0 renamed StreamingClient to RealTimeTranscriber, along with StreamingParameters to RealTimeParameters and StreamingEvents to RealTimeEvents. Use the RealTime names in new code.

What audio format works best for real-time transcription?

Send 16 kHz mono 16-bit PCM for microphone audio. Higher sample rates don’t improve accuracy. The streaming API also accepts AAC and Opus, which carry their own sample rate.

How do partial transcripts differ from final transcripts?

Partials have end_of_turn: false and update while the speaker is still talking. Finals have end_of_turn: true, are fully formatted, and never change. Each event re-transcribes the whole turn, so replace the previous partial rather than appending.

Can I stream audio from a file instead of a microphone?

Yes. The first example in this tutorial streams a hosted audio feed with no microphone. For a local recording, send it at real-time pace; sending faster than real time can reduce accuracy or close the session.

How much does real-time transcription cost?

Universal-3.6 Pro Realtime costs $0.45/hr base, billed on how long the WebSocket session stays open. Keyterms prompting is included. Add-ons such as streaming diarization (+$0.12/hr) are priced separately on the pricing page.