{
"type": "Generate",
"text": "Hi, thanks for calling. "
}{
"type": "Flush"
}{
"type": "Cancel"
}{
"type": "KeepAlive"
}{
"type": "Terminate"
}{
"type": "Begin",
"id": "10885144-0d4b-4928-9eff-ef292a352937",
"expires_at": 1789556741,
"configuration": {
"language": "english",
"sample_rate": 24000,
"encoding": "pcm_s16le",
"inactivity_timeout": 120,
"word_boundaries": true,
"voice_id": "aai-jane",
"extend_biometric_consent": false,
"word_boundaries_requested": true,
"model": "assemblyai-tts-1",
"voice_expires_at": null,
"session_ttl_sec": 3600,
"limits": {
"max_generate_text_length": 2000,
"max_control_frames_per_sec": 20,
"max_egress_audio_buffer_ms": 300000,
"text_throttle_enabled": true,
"text_throttle_max_chars_per_sec": 100,
"text_throttle_initial_grace_chars": 1500,
"text_max_buffer_chars": 8000,
"session_ttl_grace_sec": 60,
"max_message_bytes": 1048576,
"max_queued_messages": 10000,
"write_stall_timeout_sec": 30,
"inactivity_timeout_min_sec": 5,
"inactivity_timeout_max_sec": 120
},
"unknown_params": []
}
}{
"type": "Audio",
"audio": "IwB6AOMABQHHAIMAJADw/wwA",
"flush_id": 0
}{
"type": "FlushDone",
"flush_id": 0,
"audio_duration_ms": 2320
}{
"type": "WordBoundaries",
"flush_id": 0,
"audio_start_ms": 0,
"words": [
{
"word": "Hello",
"start": 265,
"end": 530,
"confidence": 0.4791773557662964
},
{
"word": "there,",
"start": 546,
"end": 712,
"confidence": 0.9253882169723511
}
]
}{
"type": "Cancelled",
"audio_duration_seconds": 0.4
}{
"type": "Termination",
"session_duration_seconds": 12,
"total_input_char_length": 96,
"audio_duration_seconds": 3.12,
"voice_expires_at": null
}{
"type": "Error",
"error_code": 3009,
"error": "Too many sessions opened for this account in the last minute (allowance 100 per minute); retry in 17 s.",
"error_type": "rate_limited",
"retryable": true,
"param": null,
"retry_after_seconds": 17
}Streaming TTS WebSocket API
Send text and receive synthesized speech. Authenticate with your API key in the Authorization header, or with the token query parameter where headers can’t be set.
{
"type": "Generate",
"text": "Hi, thanks for calling. "
}{
"type": "Flush"
}{
"type": "Cancel"
}{
"type": "KeepAlive"
}{
"type": "Terminate"
}{
"type": "Begin",
"id": "10885144-0d4b-4928-9eff-ef292a352937",
"expires_at": 1789556741,
"configuration": {
"language": "english",
"sample_rate": 24000,
"encoding": "pcm_s16le",
"inactivity_timeout": 120,
"word_boundaries": true,
"voice_id": "aai-jane",
"extend_biometric_consent": false,
"word_boundaries_requested": true,
"model": "assemblyai-tts-1",
"voice_expires_at": null,
"session_ttl_sec": 3600,
"limits": {
"max_generate_text_length": 2000,
"max_control_frames_per_sec": 20,
"max_egress_audio_buffer_ms": 300000,
"text_throttle_enabled": true,
"text_throttle_max_chars_per_sec": 100,
"text_throttle_initial_grace_chars": 1500,
"text_max_buffer_chars": 8000,
"session_ttl_grace_sec": 60,
"max_message_bytes": 1048576,
"max_queued_messages": 10000,
"write_stall_timeout_sec": 30,
"inactivity_timeout_min_sec": 5,
"inactivity_timeout_max_sec": 120
},
"unknown_params": []
}
}{
"type": "Audio",
"audio": "IwB6AOMABQHHAIMAJADw/wwA",
"flush_id": 0
}{
"type": "FlushDone",
"flush_id": 0,
"audio_duration_ms": 2320
}{
"type": "WordBoundaries",
"flush_id": 0,
"audio_start_ms": 0,
"words": [
{
"word": "Hello",
"start": 265,
"end": 530,
"confidence": 0.4791773557662964
},
{
"word": "there,",
"start": 546,
"end": 712,
"confidence": 0.9253882169723511
}
]
}{
"type": "Cancelled",
"audio_duration_seconds": 0.4
}{
"type": "Termination",
"session_duration_seconds": 12,
"total_input_char_length": 96,
"audio_duration_seconds": 3.12,
"voice_expires_at": null
}{
"type": "Error",
"error_code": 3009,
"error": "Too many sessions opened for this account in the last minute (allowance 100 per minute); retry in 17 s.",
"error_type": "rate_limited",
"retryable": true,
"param": null,
"retry_after_seconds": 17
}Optional. Authenticate with a temporary token (minted with product=tts) or an API key, for clients that cannot set the Authorization header, such as browsers. Ignored when the header is present. Mint one temporary token per connection, and never put your API key in a URL that runs on a user's device.
Optional. The voice: the library slug of an AssemblyAI voice such as aai-jane, a voice's ID (a UUID, in any letter case), or the slug of one of your account's voices, such as a cloned voice. Slugs are lowercase, at most 64 characters. When unset, the session speaks the default voice of its model in its language, aai-jane. An AssemblyAI voice the session's model does not speak in the session's language closes with 3006; a voice_id that names no voice your account can use on this host closes with 3012.
Optional. The model to run the session on, such as assemblyai-tts-1, in lowercase. When unset, the default model of the session's language in the region. With voice_id, the voice must be paired with that model (3014 otherwise). A model the region does not know closes with 3006 before Begin. See Language and model.
Optional. The language of the text you send, as its full English name, not an ISO code. Case-insensitive, echoed lowercase. The API does not detect language. Defaults to english; with a library or cloned voice that speaks one language, to that voice's language, and any other value closes the session with 3006. See Languages.
english, spanish, german, italian, portuguese, frenchOptional. Output sample rate in Hz. 24000 is the model's native rate; other rates are resampled.
8000, 16000, 22050, 24000, 44100, 48000Optional. Encoding of the audio in Audio messages: raw bytes with no header. pcm_s16le is 16-bit signed little-endian PCM; pcm_mulaw and pcm_alaw are 8-bit G.711 for telephony. The aliases mulaw and alaw are accepted and echoed in canonical form. See Audio formats on WebSocket API basics.
pcm_s16le, pcm_mulaw, pcm_alawOptional. Whether to send a WordBoundaries message with per-word timings after each request's FlushDone. Takes true, false, 1 or 0, in any case. See Word boundaries.
true, falseOptional. Seconds with no frame in either direction before the session closes with 3006 (error_type inactivity_timeout). Shortens the idle timeout every session has. An integer from 5 to 120; Begin.configuration.limits reports the accepted range.
Optional. With voice_id, by the owner of a cloned voice only, true renews the voice's biometric consent and attests renewed consent on behalf of the account that owns it. Set on a voice another account owns, the session closes with 3006 (error_type not_owner). See Renew consent.
true, falseClient message with text to speak.
Client message that concludes the current request.
Client message that stops the speech in flight, for barge-in.
Client message that restarts the idle timer.
Client message that ends the session cleanly.
Server message confirming the session opened, with its effective configuration.
Server message with a chunk of synthesized audio.
Server message confirming a request's audio has all been sent.
Server message with per-word timings for a request.
Server message answering a Cancel.
Server message with the session summary after a clean Terminate.
Server message sent before every failure close, with the reason.
Was this page helpful?