Skip to main content
A Streaming TTS session is configured entirely at connect time, through the query string, and speaks text as it arrives over one long-lived connection. Nothing can be changed afterwards: to use a different voice or sample rate, open a new session.
This page covers what every session has in common. The WebSocket API reference documents each query parameter and each frame, Message frames walks through the protocol, and the Quickstart opens a first session with an SDK.

​
Hosts and data zones

Everything on this page is the same on both hosts. See Cloud endpoints and data residency. Sessions open on the path /v1/ws, with or without one trailing slash. A request on a wrong path, such as /v1/ws/session or /v1/wss, is answered with HTTP 404 before the WebSocket upgrade, so the socket never opens and no Error frame is sent.

​
Authentication

Send your API key in the Authorization header, exactly as it appears in your dashboard. The raw key works, and so does Bearer <key>.
Browsers cannot set headers on a WebSocket handshake, so the credential may also be passed as a token query parameter. token takes either an API key or a temporary token minted on your own server with product=tts. Mint a temporary token for every connection, and keep its window short: it stays valid for any number of sessions until it expires. In anything that runs on a user’s device, use a temporary token: the query string is part of the URL, and URLs are written to server access logs, browser history and developer tools, so an API key placed there is exposed. Authenticate with a temporary token walks through the token flow. If both the header and token are present, only the header is checked. A rejected header is not retried with the query value, and a leftover Authorization header, such as a default header on your HTTP client, silently overrides a temporary token, so the shorter session duration the token asked for does not apply. Send exactly one credential. A handshake with two Authorization headers is refused with HTTP 400 before the WebSocket upgrade, so the socket never opens and no Error frame is sent.
A rejected credential still opens the socket. The failure arrives afterwards as an Error frame with code 1008, followed by a close with the same code, so bad credentials look like a stream error rather than a connection failure. Handle Error frames from the start.
1008 means the credential was rejected: missing, invalid, an expired temporary token, or an account problem such as insufficient funds. Retrying unchanged will not help; fix the credential, or mint a new token. 1011 means authentication itself failed for a temporary internal reason; reconnect and retry.

​
What happens at connect

The server runs its checks in a fixed order and sends Begin only when all of them pass:
  1. Room for the connection: 3005 with error_type at_capacity and retry_after_seconds 2 if the server your connection reached already holds all the connections it takes. This check runs before your credentials are read. Retry after the delay; the next connection can land on another server.
  2. Credentials: 1008 if rejected, 1011 on a temporary internal failure.
  3. Query parameters: 3006 if any value is invalid or if a parameter is given more than once.
  4. Your account’s limit of new sessions per minute: 3009 if your account already started as many sessions as its limit allows in the current one-minute window. Sessions already open do not count. See Limits. This check runs before any voice is looked up.
  5. The language and the voice: 3006 with error_type invalid_parameter if language is not one of the supported values, invalid_request if no model of the region speaks it (param language in both cases), or invalid_request if the session’s model does not speak the AssemblyAI voice in voice_id in that language (param voice_id). The session closes with 3005 instead if the service that routes voices cannot answer. See Voice and language. Any other voice in voice_id, such as a cloned voice, is looked up here: 3012 if no voice your account can use has that voice_id or the voice cannot be used, 3014 if it is not paired with the session’s model, 3006 for a language the voice does not speak or a model_id the region does not know, and 3006 with error_type not_owner if extend_biometric_consent is set on a voice that another account owns. See Voice cloning.
  6. Begin.
Begin is the first frame the server sends, before it reads anything from you. Wait for it before sending text. If it does not arrive, an Error frame arrives instead, and its error_code equals the close code that follows. A session can also close with 3005 and error_type at_capacity shortly after Begin, when the speech service finds no room for the session within about 8 seconds. retry_after_seconds, 2 to 5, says when to try again; see Error handling.

​
Connect parameters

The query string takes voice_id, model_id, language, sample_rate, encoding, word_boundaries, inactivity_timeout, extend_biometric_consent and token. The WebSocket API reference gives each one’s values, default and refusals; Voices and Voice cloning explain how a voice and a language pick the session’s model. The rules below apply to every parameter.
  • Integers must be bare: sample_rate=24000.0 and inactivity_timeout=30s are rejected with must be an integer.
  • Booleans are true, false, 1 or 0, in any case.
  • An empty value, such as sample_rate= with nothing after it, is treated as if the parameter were omitted, and the default applies. voice_id is the exception: an empty voice_id= closes the session with 3006 on voice_id rather than giving you the default voice.
  • Each parameter may be given once. A parameter given more than once, even with the same value or an empty one, closes the session with 3006, because it is not clear which value you meant: for example Bad connection parameters: language: must be given at most once (given 2 times).
  • Unknown parameters are accepted and ignored, and Begin.configuration.unknown_params lists their names, so a misspelled parameter such as sampel_rate shows up there instead of being silently dropped. The list is sorted and holds at most 16 names, never values; a name longer than 64 characters, or with characters other than letters, digits, _, ., -, [ and ], is listed as <invalid>.
  • A known parameter with a value outside its set closes the session with code 3006 before Begin is sent. The Error text reads Bad connection parameters: <parameter>: <reason>, for example Bad connection parameters: sample_rate: sample_rate 12345 not in supported set [8000, 16000, 22050, 24000, 44100, 48000], and the frame’s param names the parameter. voice_id is also checked before Begin, against the voices your account can use and the voices of the session’s model; see below.
  • voice is a known parameter that is always refused, so a client that still sends it gets a 3006 naming the replacement rather than a session in the default voice.

​
Inactivity timeout

Every session closes with an Error frame and code 3006 once two minutes pass with no frame in either direction: nothing from you and nothing from the server. The two minutes start when Begin is sent, so a session you open and never use closes two minutes later. Every frame starts them again: each one you send (Generate, Flush, Cancel and KeepAlive), and each one the server sends you (Audio, FlushDone and every other frame). WebSocket ping and pong frames do not count. A session that is still delivering audio to you, or still owes you a FlushDone or Cancelled, is not idle: the timer starts again once that output has been delivered. To hold an idle socket open, send KeepAlive at least once every two minutes, and only when you have nothing else to send. inactivity_timeout makes the timeout shorter, from 5 seconds up to 120. A larger value closes the session with 3006 before Begin, with the text Bad connection parameters: inactivity_timeout: inactivity_timeout must not exceed 120 seconds. Begin echoes the timeout in effect as configuration.inactivity_timeout, 120 when you did not set one, and configuration.limits.inactivity_timeout_max_sec is the largest value you may set. The Error text starts with No message received for, and the frame’s error_type is inactivity_timeout. No Termination frame is sent, and text you sent without a Flush that has not been spoken yet is discarded. This close is not a request error: nothing was wrong with what you sent. Reconnect when you have something to say.

​
Audio formats

encoding and sample_rate pick the format of every Audio frame. The audio is mono, raw codec bytes with no container and no header, and any encoding goes with any sample rate: the server resamples first and then encodes. A run of bytes lasts bytes / (sample_rate * bytes_per_sample) seconds, so 24 kHz pcm_s16le is 48,000 bytes per second and 8 kHz pcm_mulaw is 8,000. The audio_duration_ms that FlushDone reports matches that formula for the bytes you received, to within a few milliseconds, which is a cheap check that nothing was dropped.
  • Pick the format your playback path needs. 24 kHz pcm_s16le is the model’s native output and passes through unchanged; every other rate or encoding is converted on the server, which adds no measurable latency. 16 kHz pcm_s16le suits a speech pipeline that already runs at 16 kHz. Telephony asks for 8 kHz pcm_mulaw or pcm_alaw, which is what a carrier media stream expects; pace those bytes out at the wire rate, 8,000 bytes per second, rather than forwarding them as fast as they arrive.
  • Play from Begin, not from your request. Raw PCM handed to an <audio> element or a Blob plays nothing, because nothing says what rate or bit depth the bytes are in. Decode the samples yourself at configuration.sample_rate from Begin, and schedule each chunk at the end of the previous one: synthesis runs several times faster than playback, so chunks scheduled at the current time overlap. Chunk sizes vary and are not aligned to samples, so carry an odd trailing byte over to the next chunk.
  • Writing a WAV file. Prepend a header that describes the bytes, as the Quickstart program does for pcm_s16le. G.711 needs format code 7 (mu-law) or 6 (A-law) with 8 bits per sample. Players and ffmpeg open those files, but Python’s wave module does not, so ask for pcm_s16le for anything you will inspect with it. G.711 bytes written under a 16-bit PCM header play as loud noise at half the real duration.
The HTTP API adds WAV and MP3 containers; see Audio formats on HTTP API basics.

​
Read the configuration back

Begin echoes the configuration the session was actually created with, which is not always the one you asked for. It holds the effective value of every connect parameter except token and model_id, whether you sent it or not, plus what the server decided for the session: model, session_ttl_sec, the limits in force, unknown_params, and word_boundaries_requested next to word_boundaries, so a false in the latter tells apart the service’s downgrade from your own choice. The Begin message in the reference lists every key. Values are normalized in the echo: language comes back lowercase, an encoding alias such as mulaw comes back as pcm_mulaw, a voice UUID in voice_id comes back lowercase, and inactivity_timeout shows the timeout in effect, 120 when you did not set one. voice_id is the voice the session speaks: the value you sent, or the default voice’s library slug, aai-jane, when you sent none. model_id is not echoed: model names the model the session runs on, whether model_id named it or the default applied. extend_biometric_consent is true when you sent it and renewals of a cloned voice’s consent are available to your account, and false otherwise; see Renew consent. word_boundaries comes back false after you requested true when the service cannot supply word timings for the session; word_boundaries_requested then still says true. resolved_voice gives the UUID and language of the voice the session resolved, also when you named it by slug, and voice_expires_at is when a cloned voice’s consent period ends, null for an AssemblyAI voice; see the Begin frame. configuration also carries keys for options these pages do not cover, and it may gain keys over time. Read the keys you need, ignore the others, and do not validate the object against a closed set. Drive your client off this object rather than off your own request. Take the sample rate for a WAV header from here, and decide whether to enable highlighting from here.

​
Parameters that do not exist

There is no speed, pitch, volume, emotion, style, or stability parameter on the WebSocket API, and no pronunciation dictionary or SSML support on either API. The HTTP API and the OpenAI-compatible route accept speed only at its default, 1.