> ## Documentation Index > Fetch the complete documentation index at: https://assemblyai.com/docs/llms.txt > Use this file to discover all available pages before exploring further. # Stream speech with word timings > Synthesize `text` and stream the audio as [server-sent events](https://html.spec.whatwg.org/multipage/server-sent-events.html), with the time at which each word is spoken. Use it to highlight text in step with playback; when you only need the audio, [Create speech](/tts/api-reference/http-api/create-speech) is simpler and smaller. The response is `text/event-stream; charset=utf-8`. Each event is an `event:` line naming its type and one `data:` line holding one JSON object, followed by a blank line. While nothing is sent for 15 seconds, a comment line `: keepalive` arrives; ignore lines that start with `:`. There are no `id:` or `retry:` fields. The events are those of the WebSocket API, with a few HTTP fields added, and a successful stream reads: `Begin`, then one or more `Audio`, then `FlushDone`, then `WordBoundaries` (best effort, usually present), then `Termination`. As on [Create speech](/tts/api-reference/http-api/create-speech), the status and headers wait for the first audio, so every failure before it is an ordinary HTTP error; `Begin` arrives together with the first `Audio`. A failure after that point sends one `Error` event and ends the stream cleanly. A stream that ends with neither `Termination` nor `Error` was cut, by the network or by a server failure; send the request again. Text with nothing to speak answers `Begin`, `FlushDone` with `audio_duration_ms: 0` and `Termination` with `word_boundaries_status: "none"`, in a body with `Content-Length`. Audio on this route is raw `pcm_s16le`, `pcm_mulaw` or `pcm_alaw`, base64 in each `Audio` event, because the word timings sit on that audio's timeline: `mp3`, `container: "wav"`, `bit_rate` and `stream: false` are refused with `422 invalid_parameter`. Word times are integer milliseconds from the first sample of the audio. ## OpenAPI ````yaml api-reference/specs/streaming-tts-speech.yaml POST /v1/speech/events openapi: 3.0.3 info: title: AssemblyAI Streaming TTS HTTP API description: >- HTTP API of Streaming Text-to-Speech: send a text in one `POST` and read the speech back as the response body. The audio streams while it is synthesized, so the first bytes arrive long before the last; with `"stream": false` the whole clip arrives at once. `POST /v1/speech/events` streams the audio as server-sent events with word timings, and `POST /v1/openai/audio/speech` takes the request shape of the OpenAI speech API, so OpenAI SDKs and the frameworks built on them work by changing their base URL. See the [Quickstart](/tts/getting-started/quickstart) for a first request and [OpenAI compatibility](/tts/openai-compatibility) for the OpenAI-compatible route. Voices, models, audio formats and text handling are those of the [Streaming TTS WebSocket API](/tts/connect-parameters). Every error the native routes answer has the JSON body described under `ErrorResponse`, and every error of the OpenAI-compatible routes the body described under `OpenAIErrorResponse`; see [HTTP API basics](/tts/api-reference/http-api/overview). version: 1.0.0 termsOfService: https://www.assemblyai.com/legal/terms-of-service contact: name: API Support email: support@assemblyai.com url: https://www.assemblyai.com/docs/ servers: - url: https://tts.us.assemblyai.com description: US data zone x-fern-server-name: US - url: https://tts.eu.assemblyai.com description: EU data zone x-fern-server-name: EU security: - apiKey: [] tags: - name: Speech description: Text to speech in one request. - name: OpenAI compatibility description: The route and the models documents for OpenAI-shaped clients. paths: /v1/speech/events: post: tags: - Speech summary: Stream speech with word timings description: >- Synthesize `text` and stream the audio as [server-sent events](https://html.spec.whatwg.org/multipage/server-sent-events.html), with the time at which each word is spoken. Use it to highlight text in step with playback; when you only need the audio, [Create speech](/tts/api-reference/http-api/create-speech) is simpler and smaller. The response is `text/event-stream; charset=utf-8`. Each event is an `event:` line naming its type and one `data:` line holding one JSON object, followed by a blank line. While nothing is sent for 15 seconds, a comment line `: keepalive` arrives; ignore lines that start with `:`. There are no `id:` or `retry:` fields. The events are those of the WebSocket API, with a few HTTP fields added, and a successful stream reads: `Begin`, then one or more `Audio`, then `FlushDone`, then `WordBoundaries` (best effort, usually present), then `Termination`. As on [Create speech](/tts/api-reference/http-api/create-speech), the status and headers wait for the first audio, so every failure before it is an ordinary HTTP error; `Begin` arrives together with the first `Audio`. A failure after that point sends one `Error` event and ends the stream cleanly. A stream that ends with neither `Termination` nor `Error` was cut, by the network or by a server failure; send the request again. Text with nothing to speak answers `Begin`, `FlushDone` with `audio_duration_ms: 0` and `Termination` with `word_boundaries_status: "none"`, in a body with `Content-Length`. Audio on this route is raw `pcm_s16le`, `pcm_mulaw` or `pcm_alaw`, base64 in each `Audio` event, because the word timings sit on that audio's timeline: `mp3`, `container: "wav"`, `bit_rate` and `stream: false` are refused with `422 invalid_parameter`. Word times are integer milliseconds from the first sample of the audio. operationId: StreamSpeechEvents parameters: - $ref: '#/components/parameters/ClientRequestId' requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/SpeechEventsRequest' examples: default: summary: Audio and word timings in the default voice value: text: Hello there, friend. responses: '200': description: The event stream. headers: X-Request-Id: $ref: '#/components/headers/X-Request-Id' X-Client-Request-Id: $ref: '#/components/headers/X-Client-Request-Id' X-TTS-Region: $ref: '#/components/headers/X-TTS-Region' X-TTS-Model: $ref: '#/components/headers/X-TTS-Model' X-TTS-Voice-Id: $ref: '#/components/headers/X-TTS-Voice-Id' X-TTS-Voice-Expires-At: $ref: '#/components/headers/X-TTS-Voice-Expires-At' X-TTS-Language: $ref: '#/components/headers/X-TTS-Language' X-TTS-Encoding: $ref: '#/components/headers/X-TTS-Encoding' X-TTS-Sample-Rate: $ref: '#/components/headers/X-TTS-Sample-Rate' X-TTS-Container: $ref: '#/components/headers/X-TTS-Container' X-TTS-Billed-Characters: $ref: '#/components/headers/X-TTS-Billed-Characters' X-TTS-Unknown-Params: $ref: '#/components/headers/X-TTS-Unknown-Params' X-TTS-Audio-Duration-Ms: $ref: '#/components/headers/X-TTS-Audio-Duration-Ms' Server-Timing: $ref: '#/components/headers/Server-Timing' X-Accel-Buffering: $ref: '#/components/headers/X-Accel-Buffering' Cache-Control: $ref: '#/components/headers/Cache-Control' content: text/event-stream: schema: type: string description: >- Server-sent events: `Begin`, `Audio`, `FlushDone`, `WordBoundaries`, `Termination` and `Error`. Their `data` objects are described under `SpeechBeginEvent`, `SpeechAudioEvent`, `SpeechFlushDoneEvent`, `SpeechWordBoundariesEvent`, `SpeechTerminationEvent` and `SpeechErrorEvent`. example: >- event: Begin data: {"type":"Begin","id":"5e8970c3-26f7-4b8e-9c55-6c1c7f0b6d2e","expires_at":1791480600,"configuration":{"voice_id":"49c443a4-8a1d-5f7e-b70b-98bdbd9823c3","language":"english","sample_rate":24000,"encoding":"pcm_s16le","container":"raw","bit_rate":null,"transport":"http","word_boundaries":true,"model":"assemblyai-tts-1","unknown_params":[]}} event: Audio data: {"type":"Audio","audio":"AAAAAP7/AQD+/wIA","flush_id":0} event: FlushDone data: {"type":"FlushDone","flush_id":0,"audio_duration_ms":1440} event: WordBoundaries data: {"type":"WordBoundaries","flush_id":0,"audio_start_ms":0,"words":[{"word":"Hello","start":265,"end":530,"confidence":0.48},{"word":"there,","start":546,"end":712,"confidence":0.93},{"word":"friend.","start":845,"end":1126,"confidence":0.85}]} event: Termination data: {"type":"Termination","session_duration_seconds":1,"total_input_char_length":20,"audio_duration_seconds":1.44,"billed_characters":20,"word_boundaries_status":"complete"} '400': $ref: '#/components/responses/NativeBadRequest' '401': $ref: '#/components/responses/NativeUnauthorized' '402': $ref: '#/components/responses/NativePaymentRequired' '404': $ref: '#/components/responses/NativeNotFound' '405': $ref: '#/components/responses/NativeMethodNotAllowed' '408': $ref: '#/components/responses/NativeRequestTimeout' '413': $ref: '#/components/responses/NativeTooLarge' '422': $ref: '#/components/responses/NativeUnprocessable' '429': $ref: '#/components/responses/NativeRateLimited' '500': $ref: '#/components/responses/NativeInternalError' '503': $ref: '#/components/responses/NativeUnavailable' components: parameters: ClientRequestId: name: X-Client-Request-Id in: header required: false schema: type: string minLength: 1 maxLength: 512 description: >- Your own id for the request, 1 to 512 printable ASCII characters. It is echoed in the response and kept with the request's records. It need not be unique; `X-Request-Id` is the unique key. An invalid value is ignored and not echoed. schemas: SpeechEventsRequest: type: object required: - text description: >- The body of [Create speech](/tts/api-reference/http-api/create-speech) without its file formats. The audio is raw `pcm_s16le`, `pcm_mulaw` or `pcm_alaw`; `encoding` `mp3`, `container` `wav`, `bit_rate` and `stream` `false` are refused. properties: text: type: string minLength: 1 maxLength: 5000 description: >- The text to speak, 1 to 5,000 Unicode code points; see `SpeechRequest`. voice_id: type: string minLength: 1 maxLength: 64 description: >- The voice, by its library slug such as `aai-jane`, its ID or the slug of one of your account's voices; see `SpeechRequest`. model_id: type: string minLength: 1 maxLength: 48 description: The name of the model to speak with; see `SpeechRequest`. extend_biometric_consent: type: boolean default: false description: >- `true` renews the biometric consent of the cloned voice in `voice_id`, by the voice's owner only; see `SpeechRequest`. The `Termination` event carries the expiry after the renewal. language: type: string enum: - english - spanish - german - italian - portuguese - french description: >- The language to speak, as its full name, case-insensitive; see `SpeechRequest`. sample_rate: type: integer enum: - 8000 - 16000 - 22050 - 24000 - 44100 - 48000 default: 24000 description: The sample rate of the audio in Hz. encoding: type: string enum: - pcm_s16le - pcm_mulaw - pcm_alaw - mulaw - alaw default: pcm_s16le description: The encoding of the audio in the `Audio` events, case-insensitive. container: type: string enum: - raw default: raw description: >- `raw` only, case-insensitive: the events carry raw samples, and `wav` is refused. word_boundaries: type: boolean default: true description: >- `false` leaves out the `WordBoundaries` event; `Termination.word_boundaries_status` is then `disabled`. speed: type: number description: Accepted when it equals `1`; any other value is refused. example: text: Hello there, friend. ErrorResponse: type: object required: - code - message - request_id - retryable - error description: >- The body of every error of the native routes. Some errors add fields: `limit` and `limit_window` (`minute` for the requests per minute, with `limit` the account's starting limit) on `rate_limited`, and `limit` on a text or a body over its limit. additionalProperties: true properties: code: type: string description: A stable, machine-readable error code. Branch on this. message: type: string description: >- What went wrong, written for people. It may be reworded, so do not match on it. param: type: string nullable: true description: >- The field the error is about, such as `sample_rate`; `null` when it is not about one field. request_id: type: string description: >- The request's id, also in the `X-Request-Id` header. Quote it in support requests. errors: type: array nullable: true items: $ref: '#/components/schemas/ErrorItem' description: For `invalid_parameter`, every field that failed; `null` otherwise. retryable: type: boolean description: Whether the same request can succeed when you retry it later. retry_after_seconds: type: integer nullable: true description: >- Seconds to wait before retrying, also in the `Retry-After` header; `null` when there is no hint. error: type: string description: The same text as `message`. limit: type: integer description: >- The limit reached. On `rate_limited`, the account's starting requests per minute. On a text or a body over its limit, that limit. Present on those errors only. limit_window: type: string description: >- On `rate_limited`, `minute`, for the requests per minute. Present on `rate_limited` only. ErrorItem: type: object required: - message properties: message: type: string description: What is wrong with this field. param: type: string nullable: true description: The field. headers: X-Request-Id: description: >- The request's id, a UUID, on every response, errors and the answer to a browser's CORS preflight included. It is the `id` of the `Begin` event and the `request_id` of an error body. Quote it in support requests. schema: type: string X-Client-Request-Id: description: The `X-Client-Request-Id` the request sent, when it was valid. schema: type: string X-TTS-Region: description: The region that served the request, such as `us-east-1`. schema: type: string X-TTS-Model: description: >- The name of the model that spoke, such as `assemblyai-tts-1`, on every `200`. It is the model the request named, else the default model of the request's language, and the same value as `model` in the `Begin` event of `/v1/speech/events`. schema: type: string X-TTS-Voice-Id: description: >- The voice the request spoke, as `voice_id` names it and as `Begin.configuration.voice_id` reports it on `/v1/speech/events`, on every `200`. It is the `voice_id` the request sent, trimmed, with a UUID in lowercase, or the default voice's library slug, such as `aai-jane`, when the request named no voice. schema: type: string X-TTS-Voice-Expires-At: description: >- When the consent period of the request's cloned voice ends, as a Unix timestamp in seconds. Present on a `200` that spoke a cloned voice of your account with a consent period; see [Voice cloning](/tts/cloned-voices#consent-period). With `extend_biometric_consent`, it is the expiry after the renewal on a buffered response and on a response without audio, and the expiry when the request started on a streamed response, whose headers leave with the first audio. schema: type: integer X-TTS-Language: description: The language spoken, such as `english`. schema: type: string X-TTS-Encoding: description: >- The encoding of the audio, in canonical form (`mulaw` is reported as `pcm_mulaw`). schema: type: string X-TTS-Sample-Rate: description: The sample rate of the audio in Hz. schema: type: integer X-TTS-Container: description: '`raw` or `wav`. Absent on MP3.' schema: type: string X-TTS-Billed-Characters: description: >- The characters the request bills once its body is delivered, in Unicode code points of the text as sent; `0` for text with nothing to speak. A streamed response sends it with the first audio, before the outcome is known; a failure on our side that then cuts the body, or ends it with an `Error` event, makes the request free. The final count is `Termination.billed_characters` on `/v1/speech/events` and `usage.billed_characters` of `speech.audio.done`. schema: type: integer X-TTS-Unknown-Params: description: >- The names of the body fields the route ignored, sorted and comma-separated, at most 16. Present when the body had any. schema: type: string X-TTS-Audio-Duration-Ms: description: >- The duration of the audio in milliseconds. Present on buffered responses (`"stream": false`) and on responses with no audio. schema: type: integer Server-Timing: description: >- `open;dur=, model;dur=, ttfa;dur=`: `open` from receipt to the start of synthesis (authentication, validation and the voice lookup), `model` from the start of synthesis to the first audio, `ttfa` from receipt to the first audio byte sent. Browsers show it in their network panel. schema: type: string X-Accel-Buffering: description: >- `no`, on streamed responses, so that proxies pass the audio on as it arrives. schema: type: string Cache-Control: description: >- `no-store, no-transform`: the audio is never cached or recompressed on the way. schema: type: string WWW-Authenticate: description: '`Bearer realm="assemblyai"`. On every `401`.' schema: type: string Allow: description: The methods the path answers, such as `OPTIONS, POST`. schema: type: string Retry-After: description: >- Whole seconds to wait before retrying. On every `429` and `503` with an error body. schema: type: integer responses: NativeBadRequest: description: >- The body is empty, not UTF-8, not JSON, not a JSON object, or nested thousands of levels deep (`invalid_message`), or cloned voices are not available on this host (`feature_not_enabled`: `the voice catalog is not enabled in this region, so voice_id names only a preloaded AssemblyAI voice here, such as aai-jane`, or `model_id is not enabled in this region` for a request that sent `model_id` without `voice_id`). headers: X-Request-Id: $ref: '#/components/headers/X-Request-Id' content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' example: code: invalid_message message: The body must be a JSON object param: null request_id: 5e8970c3-26f7-4b8e-9c55-6c1c7f0b6d2e errors: null retryable: false retry_after_seconds: null error: The body must be a JSON object NativeUnauthorized: description: >- No credential (`missing_authorization`), or the credential was rejected (`unauthorized`): an unknown key, an expired or already used temporary token, a temporary token minted for another product, or a disabled account. headers: X-Request-Id: $ref: '#/components/headers/X-Request-Id' WWW-Authenticate: $ref: '#/components/headers/WWW-Authenticate' content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' NativePaymentRequired: description: The account has no funds for the request (`insufficient_funds`). headers: X-Request-Id: $ref: '#/components/headers/X-Request-Id' content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' NativeNotFound: description: >- A path the API does not serve (`not_found`): an unknown path, or a documented one with a trailing slash. Paths are exact; nothing redirects. headers: X-Request-Id: $ref: '#/components/headers/X-Request-Id' content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' NativeMethodNotAllowed: description: >- A method the path does not answer (`method_not_allowed`); `Allow` lists the ones it does. headers: X-Request-Id: $ref: '#/components/headers/X-Request-Id' Allow: $ref: '#/components/headers/Allow' content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' NativeRequestTimeout: description: The body did not arrive within 10 seconds (`request_timeout`). Retry. headers: X-Request-Id: $ref: '#/components/headers/X-Request-Id' content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' NativeTooLarge: description: The body is over 131,072 bytes (`request_too_large`). headers: X-Request-Id: $ref: '#/components/headers/X-Request-Id' content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' NativeUnprocessable: description: >- The request is valid JSON but cannot be served. Error codes: - `invalid_parameter`: a field is missing, of the wrong type, out of range, given twice, sent in the query string, or does not go with another field; `param` and `errors` name every failing field. Text over 5,000 characters adds `limit`. - `invalid_parameter` also answers a `language` a cloned voice does not speak on the request's model (`param` `language`). - `invalid_request`: a supported language no model of the region speaks (`param` `language`; a value outside the supported set is `invalid_parameter`), or an AssemblyAI voice the model does not speak in the language (`param` `voice_id`; the message lists the voices available by their slugs). - `model_not_found`: a `model_id` the region does not know, or, with `voice_id`, no model of the region speaks the request's language. - `voice_not_found`: no voice with this `voice_id` that your account can use on this host, such as a bare name like `jane` in place of the library slug `aai-jane`. One message covers a voice that does not exist, was deleted, is another account's private voice or belongs to the other data zone. - `voice_expired`: the cloned voice's consent period ended. - `voice_blocked`: the cloned voice is blocked. - `voice_not_paired`: the cloned voice is not paired with the request's model, the one named by `model_id` or the default model of the voice's language (`param` `model_id`). - `model_retired`: the request's model, usually the one named by `model_id`, is retired; the message names its successor. - `not_owner`: `extend_biometric_consent` on a cloned voice that another account owns (`param` `extend_biometric_consent`): `extend_biometric_consent can only be set by the voice owner; send the request without it to use this voice.` The messages of `voice_not_paired` and `model_retired` say what to do; [The texts of 3014](/tts/error-handling#the-texts-of-3014) lists them. headers: X-Request-Id: $ref: '#/components/headers/X-Request-Id' content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' examples: invalid_parameter: summary: A field out of range value: code: invalid_parameter message: >- sample_rate: sample_rate 12345 not in supported set [8000, 16000, 22050, 24000, 44100, 48000] param: sample_rate request_id: 2f0c1d9e-7b4a-4c3e-9a51-6d8e2b7f4c10 errors: - message: >- sample_rate 12345 not in supported set [8000, 16000, 22050, 24000, 44100, 48000] param: sample_rate retryable: false retry_after_seconds: null error: >- sample_rate: sample_rate 12345 not in supported set [8000, 16000, 22050, 24000, 44100, 48000] voice_not_paired: summary: A cloned voice not paired with the model value: code: voice_not_paired message: >- Voice 01a139bf-78a0-7d18-9a4e-6c0f2b8d1e37 has no ready pairing with model assemblyai-tts-1. Name a model the voice is paired with, or pair the voice with this model, wait until the pairing is ready, and try again. param: model_id request_id: 7c2e9a41-5b3d-4f8e-a6c1-0d9b8e2f4a17 errors: null retryable: false retry_after_seconds: null error: >- Voice 01a139bf-78a0-7d18-9a4e-6c0f2b8d1e37 has no ready pairing with model assemblyai-tts-1. Name a model the voice is paired with, or pair the voice with this model, wait until the pairing is ready, and try again. NativeRateLimited: description: >- The account's requests per minute are used up (`rate_limited`, the body adds `limit`, the account's starting limit, and `limit_window` `minute`). Retry after `Retry-After`. headers: X-Request-Id: $ref: '#/components/headers/X-Request-Id' Retry-After: $ref: '#/components/headers/Retry-After' content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' NativeInternalError: description: >- An unexpected server error (`internal_error`). Retry, and contact support with the `request_id` if it persists. headers: X-Request-Id: $ref: '#/components/headers/X-Request-Id' content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' NativeUnavailable: description: >- Temporarily unavailable; retry after `Retry-After`. Error codes: - `at_capacity`: no room for the request just now (2 to 5 seconds), or, for an `mp3` request, the server is encoding as many MP3 responses as it takes (1 second; ask for a pcm encoding to avoid it). - `auth_unavailable`: the credential could not be checked (1 second). - `voice_service_unavailable`: the service that routes voices could not answer (usually 1 second). - `upstream_unavailable`: the speech service failed, or produced no audio within 20 seconds (2 seconds). - `service_restart`: the server is restarting; the retry reaches another one (1 second). - `service_unavailable`: the route is switched off for now (30 seconds). headers: X-Request-Id: $ref: '#/components/headers/X-Request-Id' Retry-After: $ref: '#/components/headers/Retry-After' content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' securitySchemes: apiKey: type: apiKey in: header name: Authorization description: >- Your AssemblyAI API key, or a temporary token, in the `Authorization` header. The raw key works directly; a `Bearer ` prefix is also accepted, which is what OpenAI SDKs send. Send exactly one `Authorization` header. The HTTP routes read the credential from this header only. ````