> ## Documentation Index
> Fetch the complete documentation index at: https://assemblyai.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# OpenAI compatibility

> Point an OpenAI SDK, or a framework built on one, at Streaming TTS and keep the code you have.

Streaming TTS answers the request shape of the OpenAI speech API, `POST /audio/speech`, under the base URL `https://tts.us.assemblyai.com/v1/openai`. Point an OpenAI SDK, or a framework that is built on one, at that base URL with your AssemblyAI API key as its API key, and it speaks with AssemblyAI voices without other changes. This page is for code that already speaks the OpenAI speech API. For a new integration, the [Python and JavaScript SDKs](https://assemblyai.com/docs/tts/getting-started/quickstart.md) and the native [LiveKit](https://assemblyai.com/docs/tts/livekit.md) and [Pipecat](https://assemblyai.com/docs/tts/pipecat.md) plugins reach the whole API, word timings and barge-in on one session included.

The route is part of the [HTTP API](https://assemblyai.com/docs/tts/api-reference/http-api/overview.md) and behaves like [Create speech](https://assemblyai.com/docs/tts/api-reference/http-api/create-speech.md): the audio streams as it is synthesized, the status waits for the first audio so every earlier failure is a real HTTP status, a failure after that point cuts the connection, and the same limits and billing apply. Only the request fields and the error body follow OpenAI; [Create speech (OpenAI compatible)](https://assemblyai.com/docs/tts/api-reference/http-api/create-speech-openai.md) is the route reference.

| Data zone | Base URL |
| - | - |
| US | `https://tts.us.assemblyai.com/v1/openai` |
| EU | `https://tts.eu.assemblyai.com/v1/openai` |

## OpenAI SDKs

**Python**

```python theme={null}
import os

import httpx2
from openai import DefaultHttpx2Client, OpenAI

client = OpenAI(
    base_url="https://tts.us.assemblyai.com/v1/openai",
    api_key=os.environ["ASSEMBLYAI_API_KEY"],
    # The SDK's own pool sizes, with idle connections kept for 2 minutes
    # instead of 5 seconds; see "Keep connections between replies" below.
    http_client=DefaultHttpx2Client(
        limits=httpx2.Limits(
            max_connections=1000, max_keepalive_connections=100, keepalive_expiry=120
        )
    ),
)

with client.audio.speech.with_streaming_response.create(
    model="assemblyai-tts-en",
    voice="alloy",
    input="Hello from AssemblyAI.",
    response_format="pcm",
) as response:
    print("request id:", response.headers.get("x-request-id"))
    response.stream_to_file("hello.pcm")
```

**JavaScript**

```javascript theme={null}
import fs from "node:fs/promises";
import OpenAI from "openai";
import { Agent, fetch } from "undici";

const client = new OpenAI({
  baseURL: "https://tts.us.assemblyai.com/v1/openai",
  apiKey: process.env.ASSEMBLYAI_API_KEY,
  // Idle connections kept for 2 minutes instead of Node's 4 seconds; see
  // "Keep connections between replies" below.
  fetch,
  fetchOptions: { dispatcher: new Agent({ keepAliveTimeout: 120_000 }) },
});

const response = await client.audio.speech.create({
  model: "assemblyai-tts-en",
  voice: "alloy",
  input: "Hello from AssemblyAI.",
  response_format: "wav",
});
await fs.writeFile("hello.wav", Buffer.from(await response.arrayBuffer()));
```

The Python example is for `openai` 3, which is built on `httpx2`; with `openai` 2 or earlier, write `httpx` for `httpx2` and `DefaultHttpxClient` for `DefaultHttpx2Client`. The JavaScript example needs `npm install undici`, because the SDK takes a `dispatcher` only together with undici's own `fetch`; leave both out in a browser.

The SDKs send the key as `Authorization: Bearer <key>`, which the API accepts. `hello.pcm` holds raw 16-bit mono PCM at 24 kHz. In a browser, give the JavaScript SDK a [temporary token](https://assemblyai.com/docs/tts/authenticate-with-a-temporary-token.md) minted with `product=tts` as its `apiKey`, never your API key, and set its `dangerouslyAllowBrowser` option to `true`: without it the SDK refuses to run in a browser.

### Keep connections between replies

Build the client once per process and reuse it. Our load balancer keeps an idle connection open for an hour, but the OpenAI SDKs close it much sooner: the Python SDK after 5 seconds, and the JavaScript SDK on Node after 4, the default of Node's `fetch`. A voice agent whose replies are further apart than that opens a new connection, a TCP and a TLS handshake, before nearly every reply. The `limits` and `dispatcher` settings in the examples above keep idle connections for 2 minutes.

In the API's region, a reply on a kept connection reached its first byte about 6 ms sooner with the Python SDK, after pauses of 6 and 65 seconds (40 replies per way), and too little sooner to measure with Node. From farther away, every reconnect costs two network round trips: about 120 ms between the US west coast and `tts.us`. To open the connection before the first reply as well, call `client.models.list()` when a call starts; it reads [`GET /v1/openai/models`](https://assemblyai.com/docs/tts/api-reference/http-api/list-openai-models), which synthesizes nothing and needs no credential. See [Optimizing accuracy and latency](https://assemblyai.com/docs/tts/getting-started/optimizing-accuracy-and-latency.md) for the other habits that shorten the wait.

## Voice agent frameworks

LiveKit and Pipecat both have native AssemblyAI plugins, documented on [LiveKit](https://assemblyai.com/docs/tts/livekit.md) and [Pipecat](https://assemblyai.com/docs/tts/pipecat.md), which run on the WebSocket API with one session per call, barge-in without reconnecting and word-timed transcripts. Use those for a new agent. The OpenAI plugins below are for a pipeline already built on them, and give up the word timings.

Voice agents should ask for `pcm`: raw PCM starts playing on the first bytes and needs no decoding, while MP3 adds about 46 ms of encoder silence (at 24 kHz) to the start of every sentence.

**LiveKit**

```python theme={null}
import os

from livekit.plugins import openai

tts = openai.TTS(
    base_url="https://tts.us.assemblyai.com/v1/openai",
    api_key=os.environ["ASSEMBLYAI_API_KEY"],
    model="assemblyai-tts-en",
    voice="alloy",
    response_format="pcm",
)
```

The plugin already keeps idle connections for 2 minutes, and LiveKit opens one when the agent session starts, so it needs no change to keep connections between replies.

**Pipecat**

```python theme={null}
import os

import httpx2
from openai import DefaultAsyncHttpx2Client
from pipecat.services.openai.tts import OpenAITTSService

tts = OpenAITTSService(
    api_key=os.environ["ASSEMBLYAI_API_KEY"],
    base_url="https://tts.us.assemblyai.com/v1/openai",
    # Idle connections kept for 2 minutes instead of 5 seconds.
    http_client=DefaultAsyncHttpx2Client(
        limits=httpx2.Limits(
            max_connections=1000, max_keepalive_connections=100, keepalive_expiry=120
        )
    ),
    settings=OpenAITTSService.Settings(model="assemblyai-tts-en", voice="alloy"),
)
```

`http_client` keeps connections between replies, as in [OpenAI SDKs](#keep-connections-between-replies). Pipecat takes `http_client` from version 1.7, and `openai` 3 from 1.10; on 1.7 to 1.9, or with `openai` 2 or earlier, build it with `httpx` and `DefaultAsyncHttpxClient` instead.

Pipecat's service always asks for `pcm` at 24 kHz, and accepts only OpenAI voice names, which the route maps to AssemblyAI voices (see [Voices](#voices)).

## Request fields

`model` and `input` are required. `voice` names the voice, `response_format` the audio (`mp3` by default, `pcm` for a voice agent, `wav`, `mulaw` or `alaw`), and `stream_format: "sse"` asks for [speech events](#speech-events) instead of bytes. [Create speech (OpenAI compatible)](https://assemblyai.com/docs/tts/api-reference/http-api/create-speech-openai.md) lists every field, its values and its refusals; the sections below cover the ones that are not plain OpenAI.

### Models

| `model` | Speaks with |
| - | - |
| `assemblyai-tts-en`, `assemblyai-tts-es`, `assemblyai-tts-de`, `assemblyai-tts-it`, `assemblyai-tts-pt`, `assemblyai-tts-fr` | The default model of that language. The code also sets the language, so `assemblyai-tts-es` asks for Spanish |
| A model's name, such as `assemblyai-tts-1` | That model, as `model_id` selects it on the [WebSocket API](https://assemblyai.com/docs/tts/api-reference/websocket-api.md). The name sets no language; see [Extensions](#extensions) for how the language is chosen |

Names are case-insensitive. OpenAI's model names, such as `tts-1`, are refused with `400` and the code `model_not_found`; [List models (OpenAI compatible)](https://assemblyai.com/docs/tts/api-reference/http-api/list-openai-models.md) answers the language model names, and the `X-TTS-Model` response header reports the model that spoke.

### Voices

`voice` keeps its OpenAI name and takes one of these:

* An OpenAI voice name, in any letter case. Each one speaks with an AssemblyAI voice in its place, as the table below shows.
* The library slug of an [AssemblyAI voice](https://assemblyai.com/docs/tts/voices.md), such as `aai-jane`, in lowercase: the `voice_id` that names the voice on every other route.
* A voice's ID, a UUID. A [cloned voice](#cloned-voices) can also go in as an object.

A bare AssemblyAI voice name such as `jane` is not one of them. It is refused with `422` (`invalid_request_error`, code `invalid_parameter`, `param` `voice`), and the message names the slug to send, `aai-jane`.

| OpenAI voice | AssemblyAI voice |
| - | - |
| `alloy` | `aai-jane` |
| `ash` | `aai-michael` |
| `ballad` | `aai-charles` |
| `cedar` | `aai-george` |
| `coral` | `aai-anna` |
| `echo` | `aai-paul` |
| `fable` | `aai-charles` |
| `marin` | `aai-mary` |
| `nova` | `aai-eve` |
| `onyx` | `aai-george` |
| `sage` | `aai-vera` |
| `shimmer` | `aai-alba` |
| `verse` | `aai-michael` |

An OpenAI voice name speaks with the AssemblyAI voice in this table, which speaks English. Jane has no OpenAI name; ask for her, like any AssemblyAI voice, by her slug, `aai-jane`. The response header `X-TTS-Voice-Id` gives the `voice_id` the request spoke with, and when you sent an OpenAI voice name, `X-TTS-Voice-Alias` holds it. Other values are not supported: a misspelt name is refused with `422` like a bare name, and a slug or UUID that names no voice with `400`, code `invalid_request`, both on `voice`.

### Cloned voices

A [cloned voice](https://assemblyai.com/docs/tts/cloned-voices.md) you created in the AssemblyAI dashboard goes in `voice` as an object holding its ID or slug, `{"id": "<voice ID>"}`, which the OpenAI Python SDK passes through as it is, or as a string holding its UUID. The slug of one of your voices in a plain string is refused: send it as `{"id": "<slug>"}`. Name the model of the voice's own language, such as `assemblyai-tts-en` for an English voice, or a model's name, such as `assemblyai-tts-1`; another language's model is refused with `400` on `language`.

```python theme={null}
import os

from openai import OpenAI

client = OpenAI(
    base_url="https://tts.us.assemblyai.com/v1/openai",
    api_key=os.environ["ASSEMBLYAI_API_KEY"],
)

with client.audio.speech.with_streaming_response.create(
    model="assemblyai-tts-en",
    voice={"id": os.environ["ASSEMBLYAI_VOICE_ID"]},
    input="Thanks for calling.",
    response_format="pcm",
) as response:
    print("voice_id:", response.headers.get("x-tts-voice-id"))
    response.stream_to_file("thanks.pcm")
```

The voice's refusals answer `400` `invalid_request_error` with the code `voice_not_found`, `voice_expired`, `voice_blocked`, `voice_not_paired`, `model_retired`, `model_not_found` or `not_owner`, and `biometric_terms_not_accepted` answers `403`; see [Errors](https://assemblyai.com/docs/tts/cloned-voices.md#errors).

### Extensions

These fields are not part of the OpenAI request; most OpenAI clients pass them through an extra-body option (`extra_body` in the Python SDK):

| Field | Values |
| - | - |
| `language` | A language name (`english`) or an ISO 639-1 code (`en`), in any letter case. Refused when it differs from the language `model` implies |
| `sample_rate` | `8000`, `16000`, `22050`, `24000`, `44100`, `48000`; default `24000` |
| `voice_id` | The ID or slug of a [cloned voice](#cloned-voices), for clients that cannot send `voice` as an object. Leave `voice` out: next to any other `voice`, such as `alloy` or `aai-jane`, it is refused (`voice_id: conflicts with voice; give one of them`) |
| `extend_biometric_consent` | `true` renews the biometric consent of a cloned voice named in `voice` as `{"id": ...}` or by its UUID, or in `voice_id`, and attests renewed consent on behalf of the account that owns it. Only the voice's owner may set it; see [Renew consent](https://assemblyai.com/docs/tts/cloned-voices.md#renew-consent) |

The language is the one `language` sets, else the one the model name implies, else a cloned voice's own language, else English. Clients that cannot add fields pick a language with the model name.

```python theme={null}
import os

from openai import OpenAI

client = OpenAI(
    base_url="https://tts.us.assemblyai.com/v1/openai",
    api_key=os.environ["ASSEMBLYAI_API_KEY"],
)

with client.audio.speech.with_streaming_response.create(
    model="assemblyai-tts-en",
    voice="49c443a4-8a1d-5f7e-b70b-98bdbd9823c3",
    input="Thanks for calling.",
    response_format="pcm",
    extra_body={"language": "en", "sample_rate": 16000},
) as response:
    response.stream_to_file("thanks.pcm")
```

Word timings are not available on this route: `word_boundaries` must be left out or `false`. Use [Stream speech with word timings](https://assemblyai.com/docs/tts/api-reference/http-api/stream-speech-events.md) for them.

## Speech events

With `"stream_format": "sse"` the response is `text/event-stream` in OpenAI's form: one `data:` line per event and no `event:` lines. While nothing is sent for 15 seconds, a `: keepalive` comment line arrives; skip lines that start with `:`.

```text theme={null}
data: {"type":"speech.audio.delta","audio":"AAAAAP7/AQD+/wIA"}

data: {"type":"speech.audio.delta","audio":"/f8DAP7/AQAAAP//"}

data: {"type":"speech.audio.done","usage":{"input_tokens":0,"output_tokens":0,"total_tokens":0,"input_characters":22,"billed_characters":22,"audio_duration_seconds":1.62}}
```

* Each `speech.audio.delta` carries base64 of the next bytes of the `response_format`: with `wav`, the WAV header comes first; with `mp3`, whole MP3 frames.
* `speech.audio.done` ends the stream. Its `usage` counts characters, not tokens: the token counts are always `0`, `input_characters` is the length of `input`, and `billed_characters` what the request is billed. There is no `[DONE]` line.
* A failure after the first audio sends `{"type": "error", "error": {...}}`, with the fields of the error body below, and then cuts the connection. With `httpx`, read with `iter_bytes()` or `iter_lines()` and no `chunk_size`, or the `error` event can be lost with the cut; see [Streamed and buffered responses](https://assemblyai.com/docs/tts/api-reference/http-api/overview.md#the-status-waits-for-the-first-audio).

## Errors

Errors use OpenAI's body, so OpenAI SDKs raise their usual exceptions (`openai.BadRequestError`, `openai.RateLimitError`, ...). The `code` is the one the [HTTP API](https://assemblyai.com/docs/tts/api-reference/http-api/overview.md#error-codes) uses:

```json theme={null}
{
  "error": {
    "message": "model: 'tts-1' is not an AssemblyAI model; use one of GET /v1/openai/models, such as assemblyai-tts-en",
    "type": "invalid_request_error",
    "param": "model",
    "code": "model_not_found",
    "request_id": "93309b7e-9131-474f-99e4-6e0623463374",
    "retryable": false,
    "retry_after_seconds": null
  }
}
```

Every status and code of the route is listed on [Create speech (OpenAI compatible)](https://assemblyai.com/docs/tts/api-reference/http-api/create-speech-openai.md). In short: `400` for a bad field, model or voice, `401` for the credential, `402` for funds, `422` for a `voice` string that is not an OpenAI name, a slug or an ID, `429` for both the requests-per-minute limit (`rate_limited`) and no room just now (`at_capacity`), and `503` when the service is unavailable.

OpenAI SDKs retry a `429` on their own, twice by default. A request can be refused `at_capacity` after it was counted against your requests per minute, so each of those retries counts as a request too. To keep retries within your limit, set `max_retries` on the client and back off for `Retry-After` yourself.

Every response carries the request's id in `X-Request-Id`, and a `200` also in `X-Generation-Id`; quote it in support requests. Send `X-Client-Request-Id` (1 to 512 printable ASCII characters) to have your own id echoed back.

## Related

* [Create speech (OpenAI compatible)](https://assemblyai.com/docs/tts/api-reference/http-api/create-speech-openai.md): the route reference, with every field, status and code.
* [List models (OpenAI compatible)](https://assemblyai.com/docs/tts/api-reference/http-api/list-openai-models.md) and [List models (OpenRouter)](https://assemblyai.com/docs/tts/api-reference/http-api/list-openrouter-models.md): the public models documents.
* [HTTP API basics](https://assemblyai.com/docs/tts/api-reference/http-api/overview.md): hosts, headers, limits and billing shared with the native routes; [Word boundaries](https://assemblyai.com/docs/tts/word-boundaries.md#over-the-http-api) for word timings over HTTP.
* [LiveKit](https://assemblyai.com/docs/tts/livekit.md) and [Pipecat](https://assemblyai.com/docs/tts/pipecat.md): the native plugins.
* [Optimizing accuracy and latency](https://assemblyai.com/docs/tts/getting-started/optimizing-accuracy-and-latency.md): client habits that shorten the time to first audio.