Stream speech with word timings
Synthesize text and stream the audio as server-sent events, with the time at which each word is spoken. Use it to highlight text in step with playback; when you only need the audio, Create speech is simpler and smaller.
The response is text/event-stream; charset=utf-8. Each event is an event: line naming its type and one data: line holding one JSON object, followed by a blank line. While nothing is sent for 15 seconds, a comment line : keepalive arrives; ignore lines that start with :. There are no id: or retry: fields. The events are those of the WebSocket API, with a few HTTP fields added, and a successful stream reads:
Begin, then one or more Audio, then FlushDone, then WordBoundaries (best effort, usually present), then Termination.
As on Create speech, the status and headers wait for the first audio, so every failure before it is an ordinary HTTP error; Begin arrives together with the first Audio. A failure after that point sends one Error event and ends the stream cleanly. A stream that ends with neither Termination nor Error was cut, by the network or by a server failure; send the request again. Text with nothing to speak answers Begin, FlushDone with audio_duration_ms: 0 and Termination with word_boundaries_status: "none", in a body with Content-Length.
Audio on this route is raw pcm_s16le, pcm_mulaw or pcm_alaw, base64 in each Audio event, because the word timings sit on that audio’s timeline: mp3, container: "wav", bit_rate and stream: false are refused with 422 invalid_parameter. Word times are integer milliseconds from the first sample of the audio.
Authorizations
Your AssemblyAI API key, or a temporary token, in the Authorization header. The raw key works directly; a Bearer prefix is also accepted, which is what OpenAI SDKs send. Send exactly one Authorization header. The HTTP routes read the credential from this header only.
Headers
Your own id for the request, 1 to 512 printable ASCII characters. It is echoed in the response and kept with the request's records. It need not be unique; X-Request-Id is the unique key. An invalid value is ignored and not echoed.
1 - 512Body
The body of Create speech without its file formats. The audio is raw pcm_s16le, pcm_mulaw or pcm_alaw; encoding mp3, container wav, bit_rate and stream false are refused.
The text to speak, 1 to 5,000 Unicode code points; see SpeechRequest.
1 - 5000The voice, by its library slug such as aai-jane, its ID or the slug of one of your account's voices; see SpeechRequest.
1 - 64The name of the model to speak with; see SpeechRequest.
1 - 48true renews the biometric consent of the cloned voice in voice_id, by the voice's owner only; see SpeechRequest. The Termination event carries the expiry after the renewal.
The language to speak, as its full name, case-insensitive; see SpeechRequest.
english, spanish, german, italian, portuguese, french The sample rate of the audio in Hz.
8000, 16000, 22050, 24000, 44100, 48000 The encoding of the audio in the Audio events, case-insensitive.
pcm_s16le, pcm_mulaw, pcm_alaw, mulaw, alaw raw only, case-insensitive: the events carry raw samples, and wav is refused.
raw false leaves out the WordBoundaries event; Termination.word_boundaries_status is then disabled.
Accepted when it equals 1; any other value is refused.
Response
The event stream.
Server-sent events: Begin, Audio, FlushDone, WordBoundaries, Termination and Error. Their data objects are described under SpeechBeginEvent, SpeechAudioEvent, SpeechFlushDoneEvent, SpeechWordBoundariesEvent, SpeechTerminationEvent and SpeechErrorEvent.