How turn detection works
Streaming does not endpoint on silence alone. When a speaker pauses, the model evaluates what has been said so far to judge whether the speaker is finished or just pausing mid-thought. If the turn ends in an entity that may still continue, such as part of a phone number, the turn stays open. This is called entity-aware endpointing. Two silence thresholds control this behavior.
Lower values end turns faster. Higher values tolerate longer pauses without fragmenting the turn. Both parameters can be set on the connection and updated mid-stream, and their defaults come from the
mode preset you select.
Speaker labels change the defaultsWhen
speaker_labels is enabled, the server applies diarization-tuned defaults instead (min_turn_silence: 640, max_turn_silence: 768) so turns break cleanly at speaker changes.Entity-aware endpointing
Phone numbers, account IDs, email addresses, and spelled-out names are the hardest turns to endpoint. Speakers read them out in chunks with real pauses in between, the chunks have no grammatical ending, and the model may already have written a chunk as a finished sentence. An endpointer that ends the turn at the first pause splits the entity across several turns, and your LLM or validation logic receives fragments. Entity-aware endpointing keeps the entity in one turn. When themin_turn_silence check runs and the transcript so far ends like an entity that may still continue, the turn is held open even if it reads as a complete sentence. This covers digits, alphanumeric and spelled-out codes, email addresses and usernames, street addresses, dates, and names. If the speaker continues, the new speech stays in the same turn. If they don’t, the turn ends once the silence reaches max_turn_silence. Turns that end in an ordinary complete sentence still end at min_turn_silence.
While a turn is held, you keep receiving partial Turn messages for it, and end_of_turn_confidence rises toward 1.0 as the silence approaches max_turn_silence.
Entity-aware endpointing is on by default in every mode, and needs no parameters. Because a held turn ends at max_turn_silence, the preset you choose also sets how long the session waits for the rest of an entity: 640 ms in min_latency, 1280 ms in balanced, and 2560 ms in max_accuracy.
On our voice-agent benchmark (Universal-3.6 Pro, balanced mode), entity-aware endpointing cuts the phone-number entity error rate from 7.0% to 2.2% and leaves the median end-of-turn latency unchanged. The latency cost falls only on held turns, roughly one turn in ten, so the 90th-percentile end-of-turn latency rises from 0.83 s to 1.64 s. See the Universal-3.6 Pro Realtime research post for the full evaluation.
The lifecycle of a turn
Each turn produces a predictable message sequence of oneSpeechStarted message, one or more partial Turn messages, and a final Turn message.
Every Turn message re-transcribes the full turn audio, so each message supersedes the previous one. Always replace what you have rendered rather than appending. This also means transcripts get more accurate as the turn progresses, since each pass gives the model more audio and more context to work with. For maximum accuracy, use the final transcript marked end_of_turn: true. Use partials where responsiveness matters more than accuracy.
SpeechStarted
The server sendsSpeechStarted once per turn, immediately before the turn’s first Turn message. It carries the turn’s start timestamp in milliseconds from the start of the audio stream, and the confidence of a new turn starting.
SpeechStarted is only emitted once the model produces an actual transcript, so background noise alone does not trigger it (except occasionally on AssemblyAI-STT-3.7-Realtime, see below). That makes it a reliable barge-in signal for voice agents. When it arrives, stop your agent’s TTS playback and start listening.
Partials
While the speaker talks, the session emitsTurn messages with end_of_turn: false.
- Early partials are emitted shortly after speech begins, timed by the
interruption_delayparameter. - Pause partials are emitted when a pause triggers an end-of-turn check but the turn reads as incomplete or ends in an entity that may still continue. While an entity hold lasts, they repeat until the turn ends.
- Continuous partials are emitted roughly every 1 to 1.5 seconds during uninterrupted speech, so long utterances keep updating even without pauses.
Finals
When the turn ends, the session emits aTurn message with end_of_turn: true. The final is fully formatted, with punctuation, casing, and numbers and entities rendered in written form.
Tuning interruption sensitivity
SpeechStarted and early partials are what trigger interruptions in a voice agent, and two parameters control how eagerly they fire.
If quiet or distant speech is being missed, lower
vad_threshold so softer audio counts as speech. If background noise or the agent’s own TTS audio is triggering false interruptions, raise vad_threshold so the session requires stronger evidence of speech, increase interruption_delay so it waits longer before emitting the first partial, or enable Voice Focus to suppress background audio before it reaches the model.
Like the silence thresholds, both can be set on the connection and updated mid-stream.
Capturing entities like phone numbers
Entity-aware endpointing keeps most entities in one turn without any configuration. When your application knows an entity is coming (for example, your voice agent just asked for a callback number) and you want extra margin, raisemin_turn_silence mid-stream so that no pause ends the turn early, including a pause after a chunk that doesn’t read as an entity. Afterward, set your mode again to restore the preset’s defaults. Changes take effect immediately, without reconnecting.
- Python
- Python SDK
- Javascript
- JavaScript SDK
max_turn_silence. If speakers pause longer than that between chunks, for example while reading a card number off a card, raise max_turn_silence the same way. See Updating configuration mid-stream for the full list of mid-stream parameters.
Bring your own turn detection
If you run your own VAD or turn detection model, send aForceEndpoint message at any time during a turn to end it immediately. The server responds with the turn’s final Turn message right away.
If no words have been spoken since the last final, a final Turn message is still sent with an empty transcript.
- Python
- Python SDK
- Javascript
- JavaScript SDK
Manual mode
ForceEndpoint works in every mode, but the server will also keep ending turns on its own, based on the mode and any min_turn_silence or max_turn_silence set.
If the client should control endpointing and turn detection using an external turn detection model (e.g., LiveKit’s v1 or v1-mini turn detector) or another signal (e.g., your own VAD or a push-to-talk key being released), connect with mode set to manual.
- Python
- Javascript
ForceEndpoint to end it.
Parameter behavior
The parameters typically controlling endpointing work differently:min_turn_silence— No longer ends turns.max_turn_silence— If speech is detected and noForceEndpointis ever received, the server ends the turn after this amount of silence. Defaults to10000(ms) in manual mode (set during connection or viaUpdateConfiguration).
Turn lifecycle
Each turn follows this contract:- When the user starts speaking,
SpeechStartedis sent, followed by the turn’s first partial. - Partials keep arriving while the user speaks (roughly every second) and upon every pause in speech.
- The turn ends for one of three reasons:
ForceEndpointis sent at any point in the turn.- If
ForceEndpointis never sent, silence reachesmax_turn_silence. - If
ForceEndpointis never sent, the turn reaches its maximum duration of 60 seconds.
- Each
ForceEndpointreceives exactly one finalTurnmessage (end_of_turn: true). If no words have been spoken since the last final, itstranscriptis empty ("").
Other behavior
All other model behavior is the same, including:- Partials and finals are both formatted, but a partial’s transcript can change until the turn’s final.
- As with other modes, you can switch to or from
manualmid-stream withUpdateConfiguration. - If you connect with
speaker_labelsenabled, the session uses the speaker labels defaults instead of manual mode (note: this is a temporary limitation and will be removed in an upcoming update).
If you use a text-based turn detection model, there’s not currently a way to request a partial on demand. This will be added in a future update. In the meantime, use the formatted partials sent during speech and upon every pause.
Turn detection on AssemblyAI-STT-3.7-Realtime
assemblyai-stt-3-7-realtime is built for notetakers and ambient transcription, so it detects turns to segment the transcript rather than to drive voice-agent turn-taking. For voice agents, use AssemblyAI-STT-3.7-Realtime-Agent (see Select the speech model). It still uses content-aware endpointing, where a finished sentence ends the turn and a turn ending in an entity is held, and it responds to mode, max_turn_silence, vad_threshold, and mid-stream UpdateConfiguration. It differs from the voice-agent models in these ways:
- The end-of-turn check never runs before 320 ms of silence, so
min_turn_silencevalues below 320 have no effect. interruption_delayhas no effect, and the first partial arrives about 0.8 s after speech onset.- Mid-turn partials re-transcribe only recent audio, so earlier words in a partial can stay unchanged until the final, which re-transcribes the whole turn.
- Background noise can occasionally produce
SpeechStartedfollowed by an empty final. Clients should ignore these. - A
ForceEndpointsent when no turn is in progress gets no reply. Onassemblyai-stt-3-7-realtime-agentit gets an empty final.
Going further
- Optimizing accuracy and latency covers the
modepresets that set turn detection defaults, plus advanced tuning parameters. - Message sequence documents the complete WebSocket message contract, including turn ordering guarantees.
- Updating configuration mid-stream lists everything you can change during a live session.