Start with an SDK
The Python and JavaScript SDKs cover both protocols.synthesize() sends one text over HTTP and returns the whole audio, stream() sends one text over HTTP and yields the audio as it is made, and a session object speaks a conversation over one WebSocket connection with flush, cancel and the audio of each request in order. They also raise a refused request or a rejected session as a typed error, so you never parse an error body or frame yourself. The Quickstart shows all three, and every feature page shows its SDK form first.
Reach for the raw protocol when you work in another language, when you run in a browser or on a device without the SDK, or when you need something the SDK does not expose yet. The quickstart’s cURL tab and its direct WebSocket program are the starting points.
Whole texts over HTTP
The HTTP API turns one text into speech in one request. The audio is the response body, streamed while it is synthesized by default or as a whole file on request, and it comes as raw PCM, G.711, WAV or MP3. A second route answers the audio together with the time at which each word is spoken. Every request stands alone, a failure is an HTTP status with a JSON body, and nothing has to be kept alive between requests. This fits a voice agent that sends one request per sentence, a server that renders prompts or notifications to files, a script, and a page that calls the API with a temporary token. Keep the connection open between requests so only the first one pays for the handshake. Each request has its own character limit, which Limits lists with the others.Conversations over WebSocket
The WebSocket API holds one connection for a whole conversation. Each sentence is a request on it, text can be sent a token at a time as an LLM writes it, the server starts speaking once it holds whole sentences, and a cancel stops the request in flight for barge-in. Word boundaries arrive with the audio. The audio is raw codec bytes, PCM or G.711, with no file container, which is what a player that starts on the first bytes wants. This fits anything that talks back and forth: a voice agent, a live assistant, a dictation reader. Opening a connection per sentence pays the handshake every time and runs into the limit on new sessions, so one session per conversation is the shape to aim for. Message frames documents the protocol and Audio formats the bytes.Already on the OpenAI speech API
If your code or framework calls the OpenAI speech API, point it at the OpenAI-compatible route with your AssemblyAI API key and it speaks with AssemblyAI voices without other changes. The OpenAI SDKs work this way, and so do the OpenAI plugins of LiveKit and Pipecat, though both frameworks also have native AssemblyAI plugins that run on the WebSocket API. It is the HTTP API with OpenAI’s request fields and error body, so the native routes remain the place for WAV, G.711, buffered responses and word timings.Where to next
- Quickstart: first audio with an SDK or cURL, then a conversation over one session.
- HTTP API basics: hosts, authentication, response framing, audio formats, errors and limits of the HTTP routes.
- WebSocket API basics, Message frames and the WebSocket API reference: the WebSocket API in full.
- OpenAI compatibility: base URLs, models, voices and framework setups.
- LiveKit and Pipecat: the AssemblyAI plugins for the two frameworks.