> ## Documentation Index > Fetch the complete documentation index at: https://assemblyai.com/docs/llms.txt > Use this file to discover all available pages before exploring further. # Streaming TTS WebSocket API > Send text and receive synthesized speech. Authenticate with your API key in the `Authorization` header, or with the `token` query parameter where headers can't be set. ## AsyncAPI ````yaml api-reference/specs/streaming-tts.yaml tts id: tts title: Tts description: > Send text and receive synthesized speech. Authenticate with your API key in the `Authorization` header, or with the `token` query parameter where headers can't be set. servers: - id: usDataResidency protocol: wss host: tts.us.assemblyai.com bindings: [] variables: [] - id: euDataResidency protocol: wss host: tts.eu.assemblyai.com bindings: [] variables: [] address: /v1/ws parameters: - id: token jsonSchema: type: string description: >- Optional. Authenticate with a [temporary token](/tts/authenticate-with-a-temporary-token) (minted with `product=tts`) or an API key, for clients that cannot set the `Authorization` header, such as browsers. Ignored when the header is present. Mint one temporary token per connection, and never put your API key in a URL that runs on a user's device. description: >- Optional. Authenticate with a [temporary token](/tts/authenticate-with-a-temporary-token) (minted with `product=tts`) or an API key, for clients that cannot set the `Authorization` header, such as browsers. Ignored when the header is present. Mint one temporary token per connection, and never put your API key in a URL that runs on a user's device. type: string required: true deprecated: false - id: voice_id jsonSchema: type: string description: >- Optional. The voice: the library slug of an [AssemblyAI voice](/tts/voices) such as `aai-jane`, a voice's ID (a UUID, in any letter case), or the slug of one of your account's voices, such as a [cloned voice](/tts/cloned-voices). Slugs are lowercase, at most 64 characters. When unset, the session speaks the default voice of its model in its language, `aai-jane`. An AssemblyAI voice the session's model does not speak in the session's language closes with `3006`; a `voice_id` that names no voice your account can use on this host closes with `3012`. examples: - aai-jane - 49c443a4-8a1d-5f7e-b70b-98bdbd9823c3 description: >- Optional. The voice: the library slug of an [AssemblyAI voice](/tts/voices) such as `aai-jane`, a voice's ID (a UUID, in any letter case), or the slug of one of your account's voices, such as a [cloned voice](/tts/cloned-voices). Slugs are lowercase, at most 64 characters. When unset, the session speaks the default voice of its model in its language, `aai-jane`. An AssemblyAI voice the session's model does not speak in the session's language closes with `3006`; a `voice_id` that names no voice your account can use on this host closes with `3012`. type: string required: true deprecated: false - id: model_id jsonSchema: type: string description: >- Optional. The model to run the session on, such as `assemblyai-tts-1`, in lowercase. When unset, the default model of the session's language in the region. With `voice_id`, the voice must be paired with that model (`3014` otherwise). A model the region does not know closes with `3006` before `Begin`. See [Language and model](/tts/cloned-voices#language-and-model). examples: - assemblyai-tts-1 description: >- Optional. The model to run the session on, such as `assemblyai-tts-1`, in lowercase. When unset, the default model of the session's language in the region. With `voice_id`, the voice must be paired with that model (`3014` otherwise). A model the region does not know closes with `3006` before `Begin`. See [Language and model](/tts/cloned-voices#language-and-model). type: string required: true deprecated: false - id: language jsonSchema: type: string description: >- Optional. The language of the text you send, as its full English name, not an ISO code. Case-insensitive, echoed lowercase. The API does not detect language. Defaults to `english`; with a library or cloned voice that speaks one language, to that voice's language, and any other value closes the session with `3006`. See [Languages](/tts/voices#languages). enum: - english - spanish - german - italian - portuguese - french default: english description: >- Optional. The language of the text you send, as its full English name, not an ISO code. Case-insensitive, echoed lowercase. The API does not detect language. Defaults to `english`; with a library or cloned voice that speaks one language, to that voice's language, and any other value closes the session with `3006`. See [Languages](/tts/voices#languages). type: string required: true deprecated: false - id: sample_rate jsonSchema: type: string description: >- Optional. Output sample rate in Hz. `24000` is the model's native rate; other rates are resampled. enum: - '8000' - '16000' - '22050' - '24000' - '44100' - '48000' default: '24000' description: >- Optional. Output sample rate in Hz. `24000` is the model's native rate; other rates are resampled. type: string required: true deprecated: false - id: encoding jsonSchema: type: string description: >- Optional. Encoding of the audio in `Audio` messages: raw bytes with no header. `pcm_s16le` is 16-bit signed little-endian PCM; `pcm_mulaw` and `pcm_alaw` are 8-bit G.711 for telephony. The aliases `mulaw` and `alaw` are accepted and echoed in canonical form. See [Audio formats](/tts/connect-parameters#audio-formats) on WebSocket API basics. enum: - pcm_s16le - pcm_mulaw - pcm_alaw default: pcm_s16le description: >- Optional. Encoding of the audio in `Audio` messages: raw bytes with no header. `pcm_s16le` is 16-bit signed little-endian PCM; `pcm_mulaw` and `pcm_alaw` are 8-bit G.711 for telephony. The aliases `mulaw` and `alaw` are accepted and echoed in canonical form. See [Audio formats](/tts/connect-parameters#audio-formats) on WebSocket API basics. type: string required: true deprecated: false - id: word_boundaries jsonSchema: type: string description: >- Optional. Whether to send a `WordBoundaries` message with per-word timings after each request's `FlushDone`. Takes `true`, `false`, `1` or `0`, in any case. See [Word boundaries](/tts/word-boundaries). enum: - 'true' - 'false' default: 'true' description: >- Optional. Whether to send a `WordBoundaries` message with per-word timings after each request's `FlushDone`. Takes `true`, `false`, `1` or `0`, in any case. See [Word boundaries](/tts/word-boundaries). type: string required: true deprecated: false - id: inactivity_timeout jsonSchema: type: string description: >- Optional. Seconds with no frame in either direction before the session closes with `3006` (`error_type` `inactivity_timeout`). Shortens the idle timeout every session has. An integer from `5` to `120`; `Begin.configuration.limits` reports the accepted range. default: '120' examples: - '30' description: >- Optional. Seconds with no frame in either direction before the session closes with `3006` (`error_type` `inactivity_timeout`). Shortens the idle timeout every session has. An integer from `5` to `120`; `Begin.configuration.limits` reports the accepted range. type: string required: true deprecated: false - id: extend_biometric_consent jsonSchema: type: string description: >- Optional. With `voice_id`, by the owner of a [cloned voice](/tts/cloned-voices) only, `true` renews the voice's biometric consent and attests renewed consent on behalf of the account that owns it. Set on a voice another account owns, the session closes with `3006` (`error_type` `not_owner`). See [Renew consent](/tts/cloned-voices#renew-consent). enum: - 'true' - 'false' default: 'false' description: >- Optional. With `voice_id`, by the owner of a [cloned voice](/tts/cloned-voices) only, `true` renews the voice's biometric consent and attests renewed consent on behalf of the account that owns it. Set on a voice another account owns, the session closes with `3006` (`error_type` `not_owner`). See [Renew consent](/tts/cloned-voices#renew-consent). type: string required: true deprecated: false bindings: [] operations: - &ref_1 id: generate title: Generate description: >- Send text to speak. Text from consecutive `Generate` messages is joined exactly as sent, so include spaces yourself. Short text is held until you `Flush`; once enough complete sentences are buffered, the server starts speaking them before any `Flush`. See [Generate](/tts/message-frames#generate). type: receive messages: - &ref_13 id: generate contentType: application/json payload: - name: Generate description: Client message with text to speak. type: object properties: - name: type type: string description: Generate required: true - name: text type: string description: >- Text to speak, at most 2000 characters (Unicode code points); a longer text closes the session with `3006`. Read as written: spell out numbers, dates and symbols where the reading matters, and remove markup. Text from consecutive `Generate` messages is joined with no separator. The two halves of a surrogate pair may arrive in consecutive messages; any other unpaired surrogate closes the session with `3006`. An empty `text` does nothing but counts toward the control-frame limit. See [How text is read](/tts/prompting-guide#how-text-is-read). required: true headers: [] jsonPayloadSchema: type: object required: - type - text properties: type: type: string const: Generate x-parser-schema-id: text: type: string maxLength: 2000 description: >- Text to speak, at most 2000 characters (Unicode code points); a longer text closes the session with `3006`. Read as written: spell out numbers, dates and symbols where the reading matters, and remove markup. Text from consecutive `Generate` messages is joined with no separator. The two halves of a surrogate pair may arrive in consecutive messages; any other unpaired surrogate closes the session with `3006`. An empty `text` does nothing but counts toward the control-frame limit. See [How text is read](/tts/prompting-guide#how-text-is-read). x-parser-schema-id: x-parser-schema-id: GeneratePayload title: Generate description: Client message with text to speak. example: |- { "type": "Generate", "text": "Hi, thanks for calling. " } bindings: [] extensions: - id: x-parser-unique-object-id value: generate bindings: [] extensions: &ref_0 - id: x-parser-unique-object-id value: tts - &ref_2 id: flush title: Flush description: >- Conclude the current request. Whatever is still unspoken is synthesized, then exactly one `FlushDone` arrives. Each request concluded by a `Flush` takes the next `flush_id`, starting at `0`. You do not have to wait for one `FlushDone` before sending the next `Flush`. type: receive messages: - &ref_14 id: flush contentType: application/json payload: - name: Flush description: Client message that concludes the current request. type: object properties: - name: type type: string description: Flush required: true headers: [] jsonPayloadSchema: type: object required: - type properties: type: type: string const: Flush x-parser-schema-id: x-parser-schema-id: FlushPayload title: Flush description: Client message that concludes the current request. example: |- { "type": "Flush" } bindings: [] extensions: - id: x-parser-unique-object-id value: flush bindings: [] extensions: *ref_0 - &ref_3 id: cancel title: Cancel description: >- Barge-in. Stop the in-flight synthesis and discard everything sent but not yet heard, including pending flushes; none of them gets a `FlushDone`. The server answers with `Cancelled`. The session stays open, and the next request takes a new `flush_id`. type: receive messages: - &ref_15 id: cancel contentType: application/json payload: - name: Cancel description: Client message that stops the speech in flight, for barge-in. type: object properties: - name: type type: string description: Cancel required: true headers: [] jsonPayloadSchema: type: object required: - type properties: type: type: string const: Cancel x-parser-schema-id: x-parser-schema-id: CancelPayload title: Cancel description: Client message that stops the speech in flight, for barge-in. example: |- { "type": "Cancel" } bindings: [] extensions: - id: x-parser-unique-object-id value: cancel bindings: [] extensions: *ref_0 - &ref_4 id: keepAlive title: Keep alive description: >- Restart the idle timer, and nothing else. Send one while the session is idle, at least once per idle timeout. It is not a warm-up. type: receive messages: - &ref_16 id: keepAlive contentType: application/json payload: - name: Keep Alive description: Client message that restarts the idle timer. type: object properties: - name: type type: string description: KeepAlive required: true headers: [] jsonPayloadSchema: type: object required: - type properties: type: type: string const: KeepAlive x-parser-schema-id: x-parser-schema-id: KeepAlivePayload title: Keep Alive description: Client message that restarts the idle timer. example: |- { "type": "KeepAlive" } bindings: [] extensions: - id: x-parser-unique-object-id value: keepAlive bindings: [] extensions: *ref_0 - &ref_5 id: terminate title: Terminate description: >- End the session cleanly. Unflushed text is flushed for you and gets its own `FlushDone`, audio in flight and pending `WordBoundaries` are delivered, then `Termination` arrives and the socket closes with `1000`. Anything sent after `Terminate` is not read. type: receive messages: - &ref_17 id: terminate contentType: application/json payload: - name: Terminate Session description: Client message that ends the session cleanly. type: object properties: - name: type type: string description: Terminate required: true headers: [] jsonPayloadSchema: type: object required: - type properties: type: type: string const: Terminate x-parser-schema-id: x-parser-schema-id: TerminatePayload title: Terminate Session description: Client message that ends the session cleanly. example: |- { "type": "Terminate" } bindings: [] extensions: - id: x-parser-unique-object-id value: terminate bindings: [] extensions: *ref_0 - &ref_6 id: begin title: Begin description: >- Receive confirmation that the session opened, with the configuration it actually got. Always the first message. type: send messages: - &ref_18 id: begin contentType: application/json payload: - name: Session Begins description: >- Server message confirming the session opened, with its effective configuration. type: object properties: - name: type type: string description: Begin required: true - name: id type: string description: The session ID. Quote it in support requests. required: true - name: expires_at type: integer description: >- When the session expires, as a Unix timestamp in seconds, normally connect time plus 3600. A temporary token with a shorter `max_session_duration_seconds` brings it forward. When it passes, the server stops accepting input, speaks unflushed text, delivers the audio in flight, and closes with `3008` at most `limits.session_ttl_grace_sec` seconds later. required: true - name: configuration type: object description: >- The configuration the session was actually created with. Drive your client off this object rather than your request. It holds the effective value of every connect parameter except `token` and `model_id`, whether you sent it or not, and may carry other keys, and gain keys over time; read the keys you need and ignore the others. See [Read the configuration back](/tts/connect-parameters#read-the-configuration-back). required: true properties: - name: language type: string description: The language the session speaks, lowercase. required: false - name: sample_rate type: integer description: The output sample rate in Hz. required: false - name: encoding type: string description: The audio encoding, in canonical form. enumValues: - pcm_s16le - pcm_mulaw - pcm_alaw required: false - name: inactivity_timeout type: integer description: The idle timeout in effect, in seconds. required: false - name: word_boundaries type: boolean description: >- Whether the session sends `WordBoundaries`. `false` after a `true` request when the service cannot supply word timings for the session. required: false - name: voice_id type: string description: >- The voice the session speaks, named as you sent it (a UUID in lowercase), or the default voice's slug when you sent none. required: false - name: extend_biometric_consent type: boolean description: >- `true` when you sent it and renewals of a cloned voice's consent are available to your account. required: false - name: word_boundaries_requested type: boolean description: The `word_boundaries` you asked for. required: false - name: model type: string description: >- The name of the model the session runs on, such as `assemblyai-tts-1`. Never null. required: false - name: voice_expires_at type: integer description: >- For your own [cloned voice](/tts/cloned-voices#consent-period), when its consent period ends, as a Unix timestamp in seconds; `null` otherwise. required: false - name: resolved_voice type: object description: The voice the session resolved. required: false properties: - name: voice_id type: string description: The voice's UUID, also when you named it by slug. required: false - name: language type: string description: The voice's language. required: false - name: session_ttl_sec type: integer description: >- Seconds from connect to `expires_at`, 3600 unless a temporary token caps the session. required: false - name: limits type: object description: >- The limits in force for the session. See [Limits](/tts/limits). required: false properties: - name: max_generate_text_length type: integer description: Characters per `Generate`. required: false - name: max_control_frames_per_sec type: number description: >- Sustained `Flush`, `KeepAlive`, `Cancel` and empty `Generate` messages per second, with about 3 seconds of burst allowance, before `3010`. required: false - name: max_egress_audio_buffer_ms type: integer description: >- Milliseconds of synthesized audio waiting for delivery before `3011`. required: false - name: text_throttle_enabled type: boolean description: Whether text intake is paced. required: false - name: text_throttle_max_chars_per_sec type: number description: Sustained characters per second of text intake pacing. required: false - name: text_throttle_initial_grace_chars type: integer description: >- Characters accepted unpaced at the start of the session. required: false - name: text_max_buffer_chars type: integer description: Characters sent but not yet synthesized before `3010`. required: false - name: session_ttl_grace_sec type: integer description: >- The most seconds the session runs past `expires_at` to deliver audio in flight before `3008`. required: false - name: max_message_bytes type: integer description: >- The largest message you may send, in bytes; a larger one closes with `1009`. required: false - name: max_queued_messages type: integer description: Messages waiting to be handled before `3010`. required: false - name: write_stall_timeout_sec type: number description: >- How long one write to your socket may stay blocked before `3011`. required: false - name: inactivity_timeout_min_sec type: integer description: The smallest `inactivity_timeout` you may set. required: false - name: inactivity_timeout_max_sec type: integer description: >- The largest `inactivity_timeout` you may set, and the timeout of a session that sets none. required: false - name: unknown_params type: array description: >- The names of query parameters the server ignored, sorted, at most 16; `` stands for a name that is too long or holds unusual characters. required: false properties: - name: item type: string required: false headers: [] jsonPayloadSchema: type: object required: - type - id - expires_at - configuration properties: type: type: string const: Begin x-parser-schema-id: id: type: string description: The session ID. Quote it in support requests. x-parser-schema-id: expires_at: type: integer description: >- When the session expires, as a Unix timestamp in seconds, normally connect time plus 3600. A temporary token with a shorter `max_session_duration_seconds` brings it forward. When it passes, the server stops accepting input, speaks unflushed text, delivers the audio in flight, and closes with `3008` at most `limits.session_ttl_grace_sec` seconds later. x-parser-schema-id: configuration: type: object description: >- The configuration the session was actually created with. Drive your client off this object rather than your request. It holds the effective value of every connect parameter except `token` and `model_id`, whether you sent it or not, and may carry other keys, and gain keys over time; read the keys you need and ignore the others. See [Read the configuration back](/tts/connect-parameters#read-the-configuration-back). additionalProperties: true properties: language: type: string description: The language the session speaks, lowercase. x-parser-schema-id: sample_rate: type: integer description: The output sample rate in Hz. x-parser-schema-id: encoding: type: string enum: - pcm_s16le - pcm_mulaw - pcm_alaw description: The audio encoding, in canonical form. x-parser-schema-id: inactivity_timeout: type: integer description: The idle timeout in effect, in seconds. x-parser-schema-id: word_boundaries: type: boolean description: >- Whether the session sends `WordBoundaries`. `false` after a `true` request when the service cannot supply word timings for the session. x-parser-schema-id: voice_id: type: string description: >- The voice the session speaks, named as you sent it (a UUID in lowercase), or the default voice's slug when you sent none. x-parser-schema-id: extend_biometric_consent: type: boolean description: >- `true` when you sent it and renewals of a cloned voice's consent are available to your account. x-parser-schema-id: word_boundaries_requested: type: boolean description: The `word_boundaries` you asked for. x-parser-schema-id: model: type: string description: >- The name of the model the session runs on, such as `assemblyai-tts-1`. Never null. x-parser-schema-id: voice_expires_at: type: integer nullable: true description: >- For your own [cloned voice](/tts/cloned-voices#consent-period), when its consent period ends, as a Unix timestamp in seconds; `null` otherwise. x-parser-schema-id: resolved_voice: type: object additionalProperties: true description: The voice the session resolved. properties: voice_id: type: string nullable: true description: The voice's UUID, also when you named it by slug. x-parser-schema-id: language: type: string nullable: true description: The voice's language. x-parser-schema-id: x-parser-schema-id: session_ttl_sec: type: integer description: >- Seconds from connect to `expires_at`, 3600 unless a temporary token caps the session. x-parser-schema-id: limits: type: object description: >- The limits in force for the session. See [Limits](/tts/limits). additionalProperties: true properties: max_generate_text_length: type: integer description: Characters per `Generate`. x-parser-schema-id: max_control_frames_per_sec: type: number description: >- Sustained `Flush`, `KeepAlive`, `Cancel` and empty `Generate` messages per second, with about 3 seconds of burst allowance, before `3010`. x-parser-schema-id: max_egress_audio_buffer_ms: type: integer description: >- Milliseconds of synthesized audio waiting for delivery before `3011`. x-parser-schema-id: text_throttle_enabled: type: boolean description: Whether text intake is paced. x-parser-schema-id: text_throttle_max_chars_per_sec: type: number description: Sustained characters per second of text intake pacing. x-parser-schema-id: text_throttle_initial_grace_chars: type: integer description: Characters accepted unpaced at the start of the session. x-parser-schema-id: text_max_buffer_chars: type: integer description: Characters sent but not yet synthesized before `3010`. x-parser-schema-id: session_ttl_grace_sec: type: integer description: >- The most seconds the session runs past `expires_at` to deliver audio in flight before `3008`. x-parser-schema-id: max_message_bytes: type: integer description: >- The largest message you may send, in bytes; a larger one closes with `1009`. x-parser-schema-id: max_queued_messages: type: integer description: Messages waiting to be handled before `3010`. x-parser-schema-id: write_stall_timeout_sec: type: number description: >- How long one write to your socket may stay blocked before `3011`. x-parser-schema-id: inactivity_timeout_min_sec: type: integer description: The smallest `inactivity_timeout` you may set. x-parser-schema-id: inactivity_timeout_max_sec: type: integer description: >- The largest `inactivity_timeout` you may set, and the timeout of a session that sets none. x-parser-schema-id: x-parser-schema-id: SessionLimits unknown_params: type: array items: type: string x-parser-schema-id: description: >- The names of query parameters the server ignored, sorted, at most 16; `` stands for a name that is too long or holds unusual characters. x-parser-schema-id: x-parser-schema-id: BeginConfiguration x-parser-schema-id: BeginPayload title: Session Begins description: >- Server message confirming the session opened, with its effective configuration. example: |- { "type": "Begin", "id": "10885144-0d4b-4928-9eff-ef292a352937", "expires_at": 1789556741, "configuration": { "language": "english", "sample_rate": 24000, "encoding": "pcm_s16le", "inactivity_timeout": 120, "word_boundaries": true, "voice_id": "aai-jane", "extend_biometric_consent": false, "word_boundaries_requested": true, "model": "assemblyai-tts-1", "voice_expires_at": null, "session_ttl_sec": 3600, "limits": { "max_generate_text_length": 2000, "max_control_frames_per_sec": 20, "max_egress_audio_buffer_ms": 300000, "text_throttle_enabled": true, "text_throttle_max_chars_per_sec": 100, "text_throttle_initial_grace_chars": 1500, "text_max_buffer_chars": 8000, "session_ttl_grace_sec": 60, "max_message_bytes": 1048576, "max_queued_messages": 10000, "write_stall_timeout_sec": 30, "inactivity_timeout_min_sec": 5, "inactivity_timeout_max_sec": 120 }, "unknown_params": [] } } bindings: [] extensions: - id: x-parser-unique-object-id value: begin bindings: [] extensions: *ref_0 - &ref_7 id: audio title: Audio description: >- Receive a chunk of synthesized audio. A request's audio never interleaves with another's, and all of it arrives before that request's `FlushDone`. type: send messages: - &ref_19 id: audio contentType: application/json payload: - name: Audio description: Server message with a chunk of synthesized audio. type: object properties: - name: type type: string description: Audio required: true - name: audio type: string description: >- Base64-encoded audio in the session's `encoding` and `sample_rate`, raw codec bytes with no header. Never empty. Chunk sizes vary, so do not assume a length. See [Audio formats](/tts/connect-parameters#audio-formats) on WebSocket API basics. required: true - name: flush_id type: integer description: >- The request this audio belongs to. Starts at `0` and advances with every `Flush` and every `Cancel`, so it can skip numbers after a `Cancel`. required: true headers: [] jsonPayloadSchema: type: object required: - type - audio - flush_id properties: type: type: string const: Audio x-parser-schema-id: audio: type: string format: byte description: >- Base64-encoded audio in the session's `encoding` and `sample_rate`, raw codec bytes with no header. Never empty. Chunk sizes vary, so do not assume a length. See [Audio formats](/tts/connect-parameters#audio-formats) on WebSocket API basics. x-parser-schema-id: flush_id: type: integer description: >- The request this audio belongs to. Starts at `0` and advances with every `Flush` and every `Cancel`, so it can skip numbers after a `Cancel`. x-parser-schema-id: x-parser-schema-id: AudioPayload title: Audio description: Server message with a chunk of synthesized audio. example: |- { "type": "Audio", "audio": "IwB6AOMABQHHAIMAJADw/wwA", "flush_id": 0 } bindings: [] extensions: - id: x-parser-unique-object-id value: audio bindings: [] extensions: *ref_0 - &ref_8 id: flushDone title: Flush done description: >- Receive notice that a request's audio has all been sent. Exactly one per `Flush`, in `flush_id` order. It means the audio was delivered to your socket, not that it has finished playing. type: send messages: - &ref_20 id: flushDone contentType: application/json payload: - name: Flush Done description: Server message confirming a request's audio has all been sent. type: object properties: - name: type type: string description: FlushDone required: true - name: flush_id type: integer description: The request that is done. required: true - name: audio_duration_ms type: integer description: >- The duration of the request's audio in milliseconds; `0` when it had nothing left to speak. required: true headers: [] jsonPayloadSchema: type: object required: - type - flush_id - audio_duration_ms properties: type: type: string const: FlushDone x-parser-schema-id: flush_id: type: integer description: The request that is done. x-parser-schema-id: audio_duration_ms: type: integer description: >- The duration of the request's audio in milliseconds; `0` when it had nothing left to speak. x-parser-schema-id: x-parser-schema-id: FlushDonePayload title: Flush Done description: Server message confirming a request's audio has all been sent. example: |- { "type": "FlushDone", "flush_id": 0, "audio_duration_ms": 2320 } bindings: [] extensions: - id: x-parser-unique-object-id value: flushDone bindings: [] extensions: *ref_0 - &ref_9 id: wordBoundaries title: Word boundaries description: >- Receive per-word timings for a request, best-effort, after its `FlushDone`. It may arrive after the next request's first `Audio`, so match it by `flush_id`. A cancelled request gets none. type: send messages: - &ref_21 id: wordBoundaries contentType: application/json payload: - name: Word Boundaries description: Server message with per-word timings for a request. type: object properties: - name: type type: string description: WordBoundaries required: true - name: flush_id type: integer description: The request these timings belong to. required: true - name: audio_start_ms type: integer description: >- Where this request's audio begins on the session's continuous audio timeline, in milliseconds. Subtract it from a word's `start` and `end` to place the word within this request's own audio. required: true - name: words type: array description: >- One entry per whitespace-delimited token of the request's text, in order, with its punctuation attached. required: true properties: - name: word type: string description: The token as you sent it. required: true - name: start type: integer description: >- Start of the word in milliseconds on the session's continuous audio timeline, counted from the first audio the session delivered. required: true - name: end type: integer description: >- End of the word in milliseconds on the same timeline. Equal to `start` for a token with no sound of its own. required: true - name: confidence type: number description: >- Confidence of the word's timing, from 0 to 1; `0` for a token with no sound of its own. required: true headers: [] jsonPayloadSchema: type: object required: - type - flush_id - audio_start_ms - words properties: type: type: string const: WordBoundaries x-parser-schema-id: flush_id: type: integer description: The request these timings belong to. x-parser-schema-id: audio_start_ms: type: integer description: >- Where this request's audio begins on the session's continuous audio timeline, in milliseconds. Subtract it from a word's `start` and `end` to place the word within this request's own audio. x-parser-schema-id: words: type: array description: >- One entry per whitespace-delimited token of the request's text, in order, with its punctuation attached. items: type: object required: - word - start - end - confidence properties: word: type: string description: The token as you sent it. x-parser-schema-id: start: type: integer description: >- Start of the word in milliseconds on the session's continuous audio timeline, counted from the first audio the session delivered. x-parser-schema-id: end: type: integer description: >- End of the word in milliseconds on the same timeline. Equal to `start` for a token with no sound of its own. x-parser-schema-id: confidence: type: number description: >- Confidence of the word's timing, from 0 to 1; `0` for a token with no sound of its own. x-parser-schema-id: x-parser-schema-id: WordBoundary x-parser-schema-id: x-parser-schema-id: WordBoundariesPayload title: Word Boundaries description: Server message with per-word timings for a request. example: |- { "type": "WordBoundaries", "flush_id": 0, "audio_start_ms": 0, "words": [ { "word": "Hello", "start": 265, "end": 530, "confidence": 0.4791773557662964 }, { "word": "there,", "start": 546, "end": 712, "confidence": 0.9253882169723511 } ] } bindings: [] extensions: - id: x-parser-unique-object-id value: wordBoundaries bindings: [] extensions: *ref_0 - &ref_10 id: cancelled title: Cancelled description: >- Receive the answer to a `Cancel`, one per `Cancel` in the order sent. `Audio` that arrives between your `Cancel` and its `Cancelled` belongs to the cancelled request; drop it. type: send messages: - &ref_22 id: cancelled contentType: application/json payload: - name: Cancelled description: Server message answering a `Cancel`. type: object properties: - name: type type: string description: Cancelled required: true - name: audio_duration_seconds type: number description: >- Audio streamed for the cancelled request only, in seconds rounded to 3 decimal places; `0.0` when nothing was in flight. required: true headers: [] jsonPayloadSchema: type: object required: - type - audio_duration_seconds properties: type: type: string const: Cancelled x-parser-schema-id: audio_duration_seconds: type: number description: >- Audio streamed for the cancelled request only, in seconds rounded to 3 decimal places; `0.0` when nothing was in flight. x-parser-schema-id: x-parser-schema-id: CancelledPayload title: Cancelled description: Server message answering a `Cancel`. example: |- { "type": "Cancelled", "audio_duration_seconds": 0.4 } bindings: [] extensions: - id: x-parser-unique-object-id value: cancelled bindings: [] extensions: *ref_0 - &ref_11 id: termination title: Termination description: >- Receive the session summary after a clean `Terminate`. Not sent when the session fails or the client closes the socket. type: send messages: - &ref_23 id: termination contentType: application/json payload: - name: Session Terminated description: Server message with the session summary after a clean `Terminate`. type: object properties: - name: type type: string description: Termination required: true - name: session_duration_seconds type: integer description: The session's duration in whole seconds, truncated. required: true - name: total_input_char_length type: integer description: >- Characters of your `Generate` messages passed to synthesis, whether or not they were spoken. required: true - name: audio_duration_seconds type: number description: >- The audio delivered to you, cancelled requests' delivered audio included, in seconds rounded to 3 decimal places. required: true - name: voice_expires_at type: integer description: >- For your own [cloned voice](/tts/cloned-voices#consent-period), when its consent period ends after the session's renewal, if any, as a Unix timestamp in seconds; `null` otherwise. required: false headers: [] jsonPayloadSchema: type: object required: - type - session_duration_seconds - total_input_char_length - audio_duration_seconds additionalProperties: true properties: type: type: string const: Termination x-parser-schema-id: session_duration_seconds: type: integer description: The session's duration in whole seconds, truncated. x-parser-schema-id: total_input_char_length: type: integer description: >- Characters of your `Generate` messages passed to synthesis, whether or not they were spoken. x-parser-schema-id: audio_duration_seconds: type: number description: >- The audio delivered to you, cancelled requests' delivered audio included, in seconds rounded to 3 decimal places. x-parser-schema-id: voice_expires_at: type: integer nullable: true description: >- For your own [cloned voice](/tts/cloned-voices#consent-period), when its consent period ends after the session's renewal, if any, as a Unix timestamp in seconds; `null` otherwise. x-parser-schema-id: x-parser-schema-id: TerminationPayload title: Session Terminated description: Server message with the session summary after a clean `Terminate`. example: |- { "type": "Termination", "session_duration_seconds": 12, "total_input_char_length": 96, "audio_duration_seconds": 3.12, "voice_expires_at": null } bindings: [] extensions: - id: x-parser-unique-object-id value: termination bindings: [] extensions: *ref_0 - &ref_12 id: error title: Error description: >- Receive the reason the session is about to close. Sent immediately before every failure close the server initiates, and always the last message of the session. type: send messages: - &ref_24 id: error contentType: application/json payload: - name: Error description: Server message sent before every failure close, with the reason. type: object properties: - name: type type: string description: Error required: true - name: error_code type: integer description: >- The WebSocket close code that follows. See [Close codes](/tts/error-handling#close-codes). enumValues: - 1008 - 1011 - 1012 - 3005 - 3006 - 3008 - 3009 - 3010 - 3011 - 3012 - 3014 required: true - name: error type: string description: >- What went wrong and what to do, written for people. It may be reworded, so log it rather than matching on it. required: true - name: error_type type: string description: >- A stable, machine-readable cause, finer than the close code. New values may be added; treat an unknown one by its `error_code` and `retryable`. See [Error types](/tts/error-handling#error-types). enumValues: - unauthorized - insufficient_funds - auth_unavailable - service_restart - invalid_parameter - invalid_message - invalid_request - model_not_found - feature_not_enabled - not_owner - biometric_terms_not_accepted - inactivity_timeout - session_expired - rate_limited - input_rate_exceeded - audio_backlog - voice_not_found - voice_expired - voice_blocked - voice_not_paired - model_retired - voice_service_unavailable - at_capacity - upstream_unavailable - internal_error required: false - name: retryable type: boolean description: Whether reconnecting with the same parameters can succeed. required: false - name: param type: string description: >- The connect parameter or message field at fault, such as `sample_rate`, `voice_id` or `language`, `Generate.text`, or `type`; `null` when there is none. required: false - name: retry_after_seconds type: integer description: >- The earliest useful retry, in seconds; `null` when there is no hint. required: false headers: [] jsonPayloadSchema: type: object required: - type - error_code - error properties: type: type: string const: Error x-parser-schema-id: error_code: type: integer enum: - 1008 - 1011 - 1012 - 3005 - 3006 - 3008 - 3009 - 3010 - 3011 - 3012 - 3014 description: >- The WebSocket close code that follows. See [Close codes](/tts/error-handling#close-codes). x-parser-schema-id: error: type: string description: >- What went wrong and what to do, written for people. It may be reworded, so log it rather than matching on it. x-parser-schema-id: error_type: type: string enum: - unauthorized - insufficient_funds - auth_unavailable - service_restart - invalid_parameter - invalid_message - invalid_request - model_not_found - feature_not_enabled - not_owner - biometric_terms_not_accepted - inactivity_timeout - session_expired - rate_limited - input_rate_exceeded - audio_backlog - voice_not_found - voice_expired - voice_blocked - voice_not_paired - model_retired - voice_service_unavailable - at_capacity - upstream_unavailable - internal_error description: >- A stable, machine-readable cause, finer than the close code. New values may be added; treat an unknown one by its `error_code` and `retryable`. See [Error types](/tts/error-handling#error-types). x-parser-schema-id: retryable: type: boolean description: Whether reconnecting with the same parameters can succeed. x-parser-schema-id: param: type: string nullable: true description: >- The connect parameter or message field at fault, such as `sample_rate`, `voice_id` or `language`, `Generate.text`, or `type`; `null` when there is none. x-parser-schema-id: retry_after_seconds: type: integer nullable: true description: >- The earliest useful retry, in seconds; `null` when there is no hint. x-parser-schema-id: x-parser-schema-id: ErrorPayload title: Error description: Server message sent before every failure close, with the reason. example: |- { "type": "Error", "error_code": 3009, "error": "Too many sessions opened for this account in the last minute (allowance 100 per minute); retry in 17 s.", "error_type": "rate_limited", "retryable": true, "param": null, "retry_after_seconds": 17 } bindings: [] extensions: - id: x-parser-unique-object-id value: error bindings: [] extensions: *ref_0 sendOperations: - *ref_1 - *ref_2 - *ref_3 - *ref_4 - *ref_5 receiveOperations: - *ref_6 - *ref_7 - *ref_8 - *ref_9 - *ref_10 - *ref_11 - *ref_12 sendMessages: - *ref_13 - *ref_14 - *ref_15 - *ref_16 - *ref_17 receiveMessages: - *ref_18 - *ref_19 - *ref_20 - *ref_21 - *ref_22 - *ref_23 - *ref_24 extensions: - id: x-parser-unique-object-id value: tts securitySchemes: [] ````