Skip to main content
Detect when a speaker has finished their turn. Streaming transcription is delivered in turns. A turn is one continuous unit of speech belonging to a single speaker. Deciding when a turn has ended, called endpointing, determines how an interactive voice application feels. End turns too early and you cut speakers off mid-thought or split a complex entity like a phone number into multiple parts. End turns too late and your voice agent feels unresponsive.

How turn detection works

Streaming does not endpoint on silence alone. When a speaker pauses, the model evaluates what has been said so far to judge whether the speaker is finished or just pausing mid-thought. If the turn ends in an entity that may still continue, such as part of a phone number, the turn stays open. This is called entity-aware endpointing. Two silence thresholds control this behavior. Lower values end turns faster. Higher values tolerate longer pauses without fragmenting the turn. Both parameters can be set on the connection and updated mid-stream, and their defaults come from the mode preset you select.
Speaker labels change the defaultsWhen speaker_labels is enabled, the server applies diarization-tuned defaults instead (min_turn_silence: 640, max_turn_silence: 768) so turns break cleanly at speaker changes.

Entity-aware endpointing

Phone numbers, account IDs, email addresses, and spelled-out names are the hardest turns to endpoint. Speakers read them out in chunks with real pauses in between, the chunks have no grammatical ending, and the model may already have written a chunk as a finished sentence. An endpointer that ends the turn at the first pause splits the entity across several turns, and your LLM or validation logic receives fragments. Entity-aware endpointing keeps the entity in one turn. When the min_turn_silence check runs and the transcript so far ends like an entity that may still continue, the turn is held open even if it reads as a complete sentence. This covers digits, alphanumeric and spelled-out codes, email addresses and usernames, street addresses, dates, and names. If the speaker continues, the new speech stays in the same turn. If they don’t, the turn ends once the silence reaches max_turn_silence. Turns that end in an ordinary complete sentence still end at min_turn_silence. While a turn is held, you keep receiving partial Turn messages for it, and end_of_turn_confidence rises toward 1.0 as the silence approaches max_turn_silence. Entity-aware endpointing is on by default in every mode, and needs no parameters. Because a held turn ends at max_turn_silence, the preset you choose also sets how long the session waits for the rest of an entity: 640 ms in min_latency, 1280 ms in balanced, and 2560 ms in max_accuracy. On our voice-agent benchmark (Universal-3.6 Pro, balanced mode), entity-aware endpointing cuts the phone-number entity error rate from 7.0% to 2.2% and leaves the median end-of-turn latency unchanged. The latency cost falls only on held turns, roughly one turn in ten, so the 90th-percentile end-of-turn latency rises from 0.83 s to 1.64 s. See the Universal-3.6 Pro Realtime research post for the full evaluation.

The lifecycle of a turn

Each turn produces a predictable message sequence of one SpeechStarted message, one or more partial Turn messages, and a final Turn message. Every Turn message re-transcribes the full turn audio, so each message supersedes the previous one. Always replace what you have rendered rather than appending. This also means transcripts get more accurate as the turn progresses, since each pass gives the model more audio and more context to work with. For maximum accuracy, use the final transcript marked end_of_turn: true. Use partials where responsiveness matters more than accuracy.

SpeechStarted

The server sends SpeechStarted once per turn, immediately before the turn’s first Turn message. It carries the turn’s start timestamp in milliseconds from the start of the audio stream, and the confidence of a new turn starting.
SpeechStarted is only emitted once the model produces an actual transcript, so background noise alone does not trigger it (except occasionally on AssemblyAI-STT-3.7-Realtime, see below). That makes it a reliable barge-in signal for voice agents. When it arrives, stop your agent’s TTS playback and start listening.

Partials

While the speaker talks, the session emits Turn messages with end_of_turn: false.
Partials are emitted at three points.
  • Early partials are emitted shortly after speech begins, timed by the interruption_delay parameter.
  • Pause partials are emitted when a pause triggers an end-of-turn check but the turn reads as incomplete or ends in an entity that may still continue. While an entity hold lasts, they repeat until the turn ends.
  • Continuous partials are emitted roughly every 1 to 1.5 seconds during uninterrupted speech, so long utterances keep updating even without pauses.
Partials are suited to live captions, barge-in handling, and speculative LLM prefetching, where you want text immediately and can tolerate it being revised.

Finals

When the turn ends, the session emits a Turn message with end_of_turn: true. The final is fully formatted, with punctuation, casing, and numbers and entities rendered in written form.
The final is the most accurate transcript of the turn. Send it to your LLM, store it, and act on it whenever accuracy matters.

Tuning interruption sensitivity

SpeechStarted and early partials are what trigger interruptions in a voice agent, and two parameters control how eagerly they fire. If quiet or distant speech is being missed, lower vad_threshold so softer audio counts as speech. If background noise or the agent’s own TTS audio is triggering false interruptions, raise vad_threshold so the session requires stronger evidence of speech, increase interruption_delay so it waits longer before emitting the first partial, or enable Voice Focus to suppress background audio before it reaches the model. Like the silence thresholds, both can be set on the connection and updated mid-stream.

Capturing entities like phone numbers

Entity-aware endpointing keeps most entities in one turn without any configuration. When your application knows an entity is coming (for example, your voice agent just asked for a callback number) and you want extra margin, raise min_turn_silence mid-stream so that no pause ends the turn early, including a pause after a chunk that doesn’t read as an entity. Afterward, set your mode again to restore the preset’s defaults. Changes take effect immediately, without reconnecting.
A held entity still ends at max_turn_silence. If speakers pause longer than that between chunks, for example while reading a card number off a card, raise max_turn_silence the same way. See Updating configuration mid-stream for the full list of mid-stream parameters.

Bring your own turn detection

If you run your own VAD or turn detection model, send a ForceEndpoint message at any time during a turn to end it immediately. The server responds with the turn’s final Turn message right away. If no words have been spoken since the last final, a final Turn message is still sent with an empty transcript.

Manual mode

ForceEndpoint works in every mode, but the server will also keep ending turns on its own, based on the mode and any min_turn_silence or max_turn_silence set. If the client should control endpointing and turn detection using an external turn detection model (e.g., LiveKit’s v1 or v1-mini turn detector) or another signal (e.g., your own VAD or a push-to-talk key being released), connect with mode set to manual.
Manual mode is supported only by assemblyai-stt-3-7-realtime-agent. On other models, connecting with mode set to manual fails with a 3006 error.
In manual mode, the model’s built-in turn detection is disabled. Instead, your client decides when each turn ends and sends ForceEndpoint to end it.

Parameter behavior

The parameters typically controlling endpointing work differently:
  • min_turn_silence — No longer ends turns.
  • max_turn_silence — If speech is detected and no ForceEndpoint is ever received, the server ends the turn after this amount of silence. Defaults to 10000 (ms) in manual mode (set during connection or via UpdateConfiguration).

Turn lifecycle

Each turn follows this contract:
  1. When the user starts speaking, SpeechStarted is sent, followed by the turn’s first partial.
  2. Partials keep arriving while the user speaks (roughly every second) and upon every pause in speech.
  3. The turn ends for one of three reasons:
    1. ForceEndpoint is sent at any point in the turn.
    2. If ForceEndpoint is never sent, silence reaches max_turn_silence.
    3. If ForceEndpoint is never sent, the turn reaches its maximum duration of 60 seconds.
  4. Each ForceEndpoint receives exactly one final Turn message (end_of_turn: true). If no words have been spoken since the last final, its transcript is empty ("").
Ending the session also finalizes any open turn.
If you receive a final with a transcript followed by an empty final after sending ForceEndpoint, the turn most likely ended on max_turn_silence or the maximum turn duration (60s) just before your ForceEndpoint was received.

Other behavior

All other model behavior is the same, including:
  • Partials and finals are both formatted, but a partial’s transcript can change until the turn’s final.
  • As with other modes, you can switch to or from manual mid-stream with UpdateConfiguration.
  • If you connect with speaker_labels enabled, the session uses the speaker labels defaults instead of manual mode (note: this is a temporary limitation and will be removed in an upcoming update).
If you use a text-based turn detection model, there’s not currently a way to request a partial on demand. This will be added in a future update. In the meantime, use the formatted partials sent during speech and upon every pause.

Turn detection on AssemblyAI-STT-3.7-Realtime

assemblyai-stt-3-7-realtime is built for notetakers and ambient transcription, so it detects turns to segment the transcript rather than to drive voice-agent turn-taking. For voice agents, use AssemblyAI-STT-3.7-Realtime-Agent (see Select the speech model). It still uses content-aware endpointing, where a finished sentence ends the turn and a turn ending in an entity is held, and it responds to mode, max_turn_silence, vad_threshold, and mid-stream UpdateConfiguration. It differs from the voice-agent models in these ways:
  • The end-of-turn check never runs before 320 ms of silence, so min_turn_silence values below 320 have no effect.
  • interruption_delay has no effect, and the first partial arrives about 0.8 s after speech onset.
  • Mid-turn partials re-transcribe only recent audio, so earlier words in a partial can stay unchanged until the final, which re-transcribes the whole turn.
  • Background noise can occasionally produce SpeechStarted followed by an empty final. Clients should ignore these.
  • A ForceEndpoint sent when no turn is in progress gets no reply. On assemblyai-stt-3-7-realtime-agent it gets an empty final.

Going further