Universal-3 Pro (Streaming)

Universal-3 Pro for streaming is optimized for real-time audio utterances typically under 10 seconds, with special efficiencies built in for low-latency turn detection and voice agent workflows. It provides the highest accuracy with native multilingual code switching, entity accuracy, and prompting support.

This model is fantastic for voice agents, agent assist, and all streaming use cases that don’t require partial transcriptions for every single subword — partials are only produced during periods of silence, with at most one partial per silence period (see Partials behavior for details). Universal-3 Pro streaming delivers exceptional entity and alphanumeric accuracy, including credit card numbers, cell phone numbers, email addresses, physical addresses, and names — all with sub-300ms time to complete transcript latency.

Quickstart

Get started with Universal-3 Pro streaming using the code below. This example streams audio from your microphone and prints transcription results in real time — no custom prompt is needed, since Universal-3-Pro automatically applies a default prompt optimized for turn detection.

$
pip install "assemblyai>=1.0.0" pyaudio

1
import logging
2
from typing import Type
3
4
import assemblyai as aai
5
from assemblyai.streaming.v3 import (
6
    BeginEvent,
7
    StreamingClient,
8
    StreamingClientOptions,
9
    StreamingError,
10
    StreamingEvents,
11
    StreamingParameters,
12
    TurnEvent,
13
    TerminationEvent,
14
)
15
16
api_key = "<YOUR_API_KEY>"
17
18
logging.basicConfig(level=logging.INFO)
19
logger = logging.getLogger(__name__)
20
21
def on_begin(self: Type[StreamingClient], event: BeginEvent):
22
    print(f"Session started: {event.id}")
23
24
def on_turn(self: Type[StreamingClient], event: TurnEvent):
25
    print(f"{event.transcript} ({event.end_of_turn})")
26
27
def on_terminated(self: Type[StreamingClient], event: TerminationEvent):
28
    print(
29
        f"Session terminated: {event.audio_duration_seconds} seconds of audio processed"
30
    )
31
32
def on_error(self: Type[StreamingClient], error: StreamingError):
33
    print(f"Error occurred: {error}")
34
35
def main():
36
    client = StreamingClient(
37
        StreamingClientOptions(
38
            api_key=api_key,
39
            api_host="streaming.assemblyai.com",
40
        )
41
    )
42
43
    client.on(StreamingEvents.Begin, on_begin)
44
    client.on(StreamingEvents.Turn, on_turn)
45
    client.on(StreamingEvents.Termination, on_terminated)
46
    client.on(StreamingEvents.Error, on_error)
47
48
    client.connect(
49
        StreamingParameters(
50
            sample_rate=16000,
51
            speech_model="u3-rt-pro",
52
        )
53
    )
54
55
    try:
56
        client.stream(
57
            aai.extras.MicrophoneStream(sample_rate=16000)
58
        )
59
    finally:
60
        client.disconnect(terminate=True)
61
62
if __name__ == "__main__":
63
    main()

Prompting

Universal-3-Pro supports custom prompts and keyterms prompting to improve transcription accuracy for your use case. For detailed guidance on crafting effective prompts, default prompt behavior, and keyterms prompting, see the Prompting Guide (Streaming).

You can also boost recognition of specific terms using the keyterms_prompt parameter. See Keyterms prompting for details.

Configuring turn detection

Universal-3-Pro uses a punctuation-based turn detection system controlled by two parameters:

When silence reaches min_turn_silence, the model transcribes the audio and checks for terminal punctuation (. ? !):

• Terminal punctuation found — the turn ends and is emitted as a final transcript (end_of_turn: true).
• No terminal punctuation — a partial transcript is emitted (end_of_turn: false) and the turn continues waiting.
  • If silence continues to max_turn_silence, the turn is forced to end as a final transcript (end_of_turn: true) regardless of punctuation.

For example, to configure both parameters:

1
{
2
  "speech_model": "u3-rt-pro",
3
  "min_turn_silence": 100,
4
  "max_turn_silence": 1200
5
}

Partials behavior

Partials are Turn events where end_of_turn is false. They are produced whenever min_turn_silence is met, but the ending punctuation doesn’t signal the end of a turn.

There can be multiple partial transcripts per turn, but each period of silence can produce at most one partial. If silence exceeds min_turn_silence, but speech resumes before max_turn_silence, the partial is emitted and the EOT check resets until the next period of silence.

If you’re running eager LLM inference on partial transcripts, we recommend setting min_turn_silence to 100.

Formatting and turn detection

Because the model applies punctuation and formatting intelligently, this works well with formatting-based turn detection. For example, based purely on vocal tone:

• "Pizza." — Statement
• "Pizza?" — Questioning tone
• "Pizza---" — Trailing off

The punctuation quality has been excellent when paired with custom turn detection models.

From testing, mid-turn emission looks like this — where each line is an additional partial leading up to the final end-of-turn transcript:

"Yeah my credit card number is--"
"One moment---"
"Its 8888-8888-8888-8888"  ← end_of_turn: true

Each partial is emitted during a silence period within the turn. The final line with terminal punctuation triggers the end of turn.

Forcing a turn endpoint

You can force the current turn to end immediately by sending a ForceEndpoint message:

1
{
2
  "type": "ForceEndpoint"
3
}

This is useful when your application knows the user has finished speaking based on external signals (e.g., a button press).

Updating configuration mid-stream

You can update configuration during an active streaming session using UpdateConfiguration. This applies changes without needing to reconnect. The recommended approach is to dynamically update keyterms_prompt based on the current stage of your voice agent flow — if you expect certain answers or terminology at a specific stage, proactively add those as keyterms so the model recognizes them accurately.

1
# Replace or establish new set of keyterms
2
websocket.send('{"type": "UpdateConfiguration", "keyterms_prompt": ["Universal-3"]}')
3
4
# Remove keyterms and reset context biasing
5
websocket.send('{"type": "UpdateConfiguration", "keyterms_prompt": []}')

For example, if your voice agent is currently asking for the caller’s name and date of birth, send the expected terms for that stage:

1
# Caller identification stage
2
websocket.send('{"type": "UpdateConfiguration", "keyterms_prompt": ["Kelly Byrne-Donoghue", "date of birth", "January", "February"]}')

Then, when the conversation moves to a different stage (e.g., medical intake), update with the relevant terms:

1
# Medical intake stage
2
websocket.send('{"type": "UpdateConfiguration", "keyterms_prompt": ["cardiology", "echocardiogram", "Dr. Patel", "metoprolol"]}')

You can also update prompt, max_turn_silence, min_turn_silence, or any combination at the same time:

1
{
2
  "type": "UpdateConfiguration",
3
  "keyterms_prompt": ["account number", "routing number"],
4
  "max_turn_silence": 5000,
5
  "min_turn_silence": 200
6
}

Common reasons to update configuration mid-stream:

• keyterms_prompt — Dynamically add terms relevant to the current stage of your voice agent flow. This is the most effective way to improve recognition accuracy mid-stream. See Keyterms prompting for details.
• prompt — Pass updated behavioral or formatting instructions into the STT stream.
• max_turn_silence — Increase for moments where you’d expect a longer pause, such as when a caller is reading out a credit card number, ID number, or address. Decrease it again afterward to resume snappier turn detection.
• min_turn_silence — Tune how quickly speculative EOT checks fire. Lower values produce faster partials for eager LLM inference, while higher values reduce entity splitting for utterances with numbers or proper nouns.