# AssemblyAI Documentation: Text to Speech (part 1 of 4)
Get an API key: https://www.assemblyai.com/dashboard/signup?utm_source=llms_full_txt&utm_medium=ai_agent&utm_campaign=agent_docs
Full Markdown for the Text to Speech section. Index of all sections: https://www.assemblyai.com/docs/llms.txt
---
# Quickstart
URL: https://www.assemblyai.com/docs/tts/getting-started/quickstart
Source: docs/tts/getting-started/quickstart.mdx
Navigation: Text to Speech > Getting started
Description: Turn a line of text into a WAV file with one SDK call or one HTTP request, then speak a whole reply over a single WebSocket session.
The Streaming TTS API turns text into speech and sends the audio back as it is synthesized, so playback can start long before the sentence ends. This page gets you from an empty file to playable audio twice: first as a WAV file from one request, then as a conversation spoken over one WebSocket session, the way a voice agent uses it.
## Before you begin
- **An API key.** Copy it from API Keys (`https://www.assemblyai.com/dashboard/signup?utm_source=docs_md&utm_medium=ai_agent&utm_campaign=agent_docs`) and set it once:
```bash
export ASSEMBLYAI_API_KEY=
```
- **The SDK.** Skip this if you only want the `curl` command. Text to speech is in the Python SDK from release 1.8.2 and in the JavaScript SDK from 4.42.0.
```bash
pip install -U assemblyai
```
```bash
npm install assemblyai
```
## Generate your first audio
One request, one text, the whole audio back as a WAV file. The SDKs call the [HTTP API](/tts/api-reference/http-api/overview), and the cURL tab shows the same request without an SDK. Save the program as `hello.py` or `hello.mjs`; the `.mjs` extension makes it an ES module, which the `import` and the top-level `await` need.
```python
import os
import assemblyai as aai
tts = aai.TextToSpeech(api_key=os.environ["ASSEMBLYAI_API_KEY"])
response = tts.synthesize(
"Hi, thanks for calling. How can I help you today?",
aai.TTSConfig(voice_id="aai-kyla", container=aai.TTSContainer.wav),
)
with open("hello.wav", "wb") as f:
f.write(response.audio)
print(f"voice {response.voice_id}, model {response.model}, "
f"{response.sample_rate} Hz {response.encoding}")
print(f"wrote hello.wav: {response.audio_duration_ms} ms of audio, "
f"{response.billed_characters} characters billed")
```
```javascript
import { writeFileSync } from "node:fs";
import { AssemblyAI } from "assemblyai";
const client = new AssemblyAI({ apiKey: process.env.ASSEMBLYAI_API_KEY });
const response = await client.tts.synthesize({
text: "Hi, thanks for calling. How can I help you today?",
voice_id: "aai-kyla",
container: "wav",
});
writeFileSync("hello.wav", response.audio);
console.log(`voice ${response.voiceId}, model ${response.model}, ${response.sampleRate} Hz ${response.encoding}`);
console.log(`wrote hello.wav: ${response.audioDurationMs} ms of audio, ${response.billedCharacters} characters billed`);
```
```bash
curl -sS --fail-with-body -D headers.txt https://tts.us.assemblyai.com/v1/speech \
-H "Authorization: $ASSEMBLYAI_API_KEY" \
-H "Content-Type: application/json" \
-d '{"text": "Hi, thanks for calling. How can I help you today?", "voice_id": "aai-kyla", "container": "wav", "stream": false}' \
-o hello.wav
```
`"stream": false` asks for the whole file at once, which is what the SDKs' `synthesize()` sends; leave it out and the audio streams as it is made. The response headers, saved in `headers.txt`, carry what the SDKs print, among others:
```text
HTTP/2 200
content-type: audio/wav
x-request-id: 83c9fae1-e0b0-433a-b97e-26f372778f2a
x-tts-model: assemblyai-tts-1
x-tts-voice-id: aai-kyla
x-tts-language: english
x-tts-encoding: pcm_s16le
x-tts-sample-rate: 24000
x-tts-container: wav
x-tts-billed-characters: 49
x-tts-audio-duration-ms: 3200
```
A refused request, a wrong key for example, exits with a non-zero code and writes the JSON error into `hello.wav` in place of the audio. If the file will not play, read it.
Run it with `python hello.py` or `node hello.mjs`, and `hello.wav` plays in any audio player. The SDKs print this:
```text
voice aai-kyla, model assemblyai-tts-1, 24000 Hz pcm_s16le
wrote hello.wav: 3200 ms of audio, 49 characters billed
```
Four things about this run:
- **Only the text is required.** `voice_id` names Kyla here by her library slug. Leave it out and you get the [default voice](/tts/voices#the-default-voice), Kyla in English, at 24 kHz 16-bit PCM. A voice speaks the language it was recorded in unless you set `language` to another one, though results are usually [worse than in the voice's own language](/tts/voices#voice-and-language). [Voices](/tts/voices) explains how to name a voice and set the language, and the same `voice_id` takes a voice you [cloned](/tts/cloned-voices) in the dashboard.
- **The response says what it ran with.** The voice, model, language, encoding and sample rate come back as response headers, which the SDKs expose on the result. Read the sample rate from there rather than from your own request when you write a header or set up a player.
- **The duration varies between runs.** Synthesis is not deterministic, so the same sentence can come back a few hundred milliseconds longer or shorter. Billing counts characters, not seconds.
- **The JavaScript SDK uses two spellings.** `synthesize()` and `stream()` take the API's own field names, `voice_id` and `sample_rate`, while `session()` takes camelCase options, `voiceId` and `sampleRate`. Results are camelCase, and the `Begin` and `Termination` frames keep their wire names.
The requests above go to the US endpoint, which processes your text and audio in the US. To keep processing in the EU, point the SDK or the URL at `tts.eu.assemblyai.com` instead. See [Cloud endpoints and data residency](/tts/endpoints-and-data-zones).
## Speak a conversation over one session
A voice agent speaks many short texts, one per turn, and needs the first audio of each as fast as possible. The WebSocket API is built for that: one connection stays open for the whole conversation, each sentence is a request on it, and the audio of a request streams back while the request is still being spoken. Opening a connection per sentence pays the handshake every time and quickly hits the per-account limit on new sessions per minute.
The programs below open a session, speak a four-sentence reply through it and write the raw audio to a file. Save them as `reply.py` or `reply.mjs`.
```python
import os
import assemblyai as aai
config = aai.TTSConfig(voice_id="aai-kyla", language="english")
reply = [
"Hi, thanks for calling.",
"You have reached the accounts team.",
"There is one open invoice from last month.",
"Would you like to pay it today?",
]
with aai.TTSSession(config, api_key=os.environ["ASSEMBLYAI_API_KEY"]) as session:
print(f"session {session.begin.id}")
with open("reply.pcm", "wb") as out:
for sentence in reply:
n = 0
for chunk in session.speak(sentence): # raw 24 kHz 16-bit mono PCM
out.write(chunk)
n += len(chunk)
print(f"{sentence!r}: {n} bytes")
summary = session.termination # None if the summary did not arrive in time
if summary:
print(f"{summary.audio_duration_seconds} s of audio "
f"for {summary.total_input_char_length} characters")
```
```javascript
import { createWriteStream } from "node:fs";
import { AssemblyAI } from "assemblyai";
const client = new AssemblyAI({ apiKey: process.env.ASSEMBLYAI_API_KEY });
const reply = [
"Hi, thanks for calling.",
"You have reached the accounts team.",
"There is one open invoice from last month.",
"Would you like to pay it today?",
];
const session = client.tts.session({ voiceId: "aai-kyla", language: "english" });
const begin = await session.connect();
console.log(`session ${begin.id}`);
const out = createWriteStream("reply.pcm");
for (const sentence of reply) {
let bytes = 0;
for await (const chunk of session.speak(sentence)) { // raw 24 kHz 16-bit mono PCM
out.write(chunk);
bytes += chunk.byteLength;
}
console.log(`'${sentence}': ${bytes} bytes`);
}
out.end();
const summary = await session.close(); // undefined if the summary did not arrive in time
if (summary) {
console.log(`${summary.audio_duration_seconds} s of audio for ${summary.total_input_char_length} characters`);
}
```
Run it with `python reply.py` or `node reply.mjs`, then play the file. It is raw PCM with no header, so tell the player its format: with [SoX](https://sourceforge.net/projects/sox/), with [ffmpeg](https://ffmpeg.org/), or by converting it to a WAV file any player opens.
```bash
play -t raw -r 24000 -e signed -b 16 -c 1 reply.pcm # SoX
ffplay -f s16le -ar 24000 -ac 1 reply.pcm # ffmpeg
sox -t raw -r 24000 -e signed -b 16 -c 1 reply.pcm reply.wav # SoX, to a WAV file
```
The output looks like this, with byte counts that differ a little from run to run:
```text
session 285ab483-997d-439d-a207-411b373a4ae1
'Hi, thanks for calling.': 88320 bytes
'You have reached the accounts team.': 99840 bytes
'There is one open invoice from last month.': 134400 bytes
'Would you like to pay it today?': 84480 bytes
8.48 s of audio for 131 characters
```
What the session did for you:
- **The server speaks first.** Connecting waits for the `Begin` frame, which echoes the configuration the session actually got, with the session `id` to quote in support requests. A rejected key or parameter, an unknown `voice_id` included, arrives as an `Error` frame in its place: the Python `with` and the JavaScript `connect()` raise it. See [Error handling](/tts/error-handling).
- **Each sentence is one request.** `speak()` sends the text in `Generate` frames, concludes the request with a `Flush` and yields the audio frames until the request's `FlushDone`. The iterator ends with the request, which is what lets the loop count bytes per sentence. See [Message frames](/tts/message-frames).
- **The audio has no header.** A session streams raw codec bytes, 16-bit PCM here, so the file will not play if you hand it to an `` element or a `Blob`. Write a WAV header yourself, or decode the samples and play them as they arrive. Chunk sizes vary, and each chunk holds whole samples. See [Audio formats](/tts/connect-parameters#audio-formats) on WebSocket API basics.
- **Leaving the session ends it cleanly.** Exiting the Python block and awaiting `close()` in JavaScript send `Terminate`, which speaks any text not yet flushed, and wait for the `Termination` summary: the seconds of audio and the characters of text that reached synthesis.
A session lasts at most an hour, and the server closes it after two minutes without a frame in either direction, so an agent waiting on a silent caller sends `keep_alive()` (`keepAlive()` in JavaScript) now and then. Keep the number of requests you have sent but not yet read bounded: the server accepts text faster than it speaks it, but only so far ahead, and a session that keeps sending far ahead of its speech is closed. The JavaScript SDK's `speak()` refuses a single text longer than that backlog before sending any of it. See [Limits](/tts/limits).
### Stream text in as an LLM writes it
Text that arrives a token at a time does not have to wait for the end of the sentence. `generate()` appends text to the current request, `flush()` concludes it and returns its `flush_id`, and `audio()` yields the request's audio. The server starts speaking once it holds enough complete sentences, before the flush, and a `flush()` at the end of a sentence starts that sentence's audio at once; [Optimizing accuracy and latency](/tts/getting-started/optimizing-accuracy-and-latency#send-text-as-it-is-written-and-flush-each-sentence) explains when to flush. `cancel()` is the barge-in: it stops the request in flight and drops anything queued.
The fragments below continue the programs above: the imports and `config` in Python, `client` in JavaScript. You supply the three functions they call: `llm_tokens()` yields the text as your LLM produces it, `user_started_talking()` says whether to interrupt, and `play()` takes a chunk of audio. An LLM client that streams asynchronously pairs with the asyncio twin, `aai.AsyncTTSSession`, whose methods are awaited.
```python
# Fragment: continues reply.py. You supply llm_tokens(), user_started_talking() and play().
with aai.TTSSession(config, api_key=os.environ["ASSEMBLYAI_API_KEY"]) as session:
for token in llm_tokens():
session.generate(token) # joined exactly as sent: include the spaces
flush_id = session.flush()
for chunk in session.audio(flush_id):
if user_started_talking():
session.cancel() # ends this iteration at once; the server drops the rest
break
play(chunk)
```
```javascript
// Fragment: continues reply.mjs. You supply llmTokens(), userStartedTalking() and play().
const session = client.tts.session({ voiceId: "aai-kyla" });
await session.connect();
for await (const token of llmTokens()) session.generate(token); // joined as sent: include the spaces
const flushId = session.flush();
for await (const chunk of session.audio(flushId)) {
if (userStartedTalking()) {
session.cancel(); // ends this iteration at once; the server drops the rest
break;
}
play(chunk);
}
await session.close();
```
Consecutive texts are joined exactly as sent, with no separator: `Good` followed by `morning.` is spoken as `Goodmorning.`, so keep the space at the edge of each piece. Requests can be pipelined, a `flush()` before the previous request's audio has been read, and their audio comes back in order. See [Generate](/tts/message-frames#generate) and [Cancel](/tts/message-frames#cancel).
## Using the WebSocket API directly
Not using an SDK? Connect to `wss://tts.us.assemblyai.com/v1/ws` with any WebSocket library, with your API key in the `Authorization` header and the voice and audio options as query parameters, and handle the frames yourself. The program below needs `websockets` 14 or newer (`pip install -U websockets`). Saved as `ws.py` and run with `python ws.py`, it reads `Begin`, sends one `Generate` and a `Flush`, collects the `Audio` frames until `FlushDone`, then terminates and writes a WAV file.
```python expandable
import asyncio
import base64
import json
import os
import struct
import websockets
URL = (
"wss://tts.us.assemblyai.com/v1/ws"
"?voice_id=aai-kyla&language=english"
"&sample_rate=24000&encoding=pcm_s16le"
)
def wav_header(n_bytes, rate, channels=1, bits=16):
byte_rate = rate * channels * bits // 8
return (
b"RIFF" + struct.pack("
# Optimizing Accuracy and Latency
URL: https://www.assemblyai.com/docs/tts/getting-started/optimizing-accuracy-and-latency
Source: docs/tts/getting-started/optimizing-accuracy-and-latency.mdx
Navigation: Text to Speech > Getting started
Description: Get the first audio of each reply sooner, and have the text read the way you mean it.
A voice agent is judged on how long the user waits for a reply to start, and on whether the reply says what you meant. Synthesis itself runs on our servers. Most of what you can change sits around it, in how you connect, how you send the text and how you write it. This page covers the choices that matter, and what each one saves.
## Measure the time to first audio
Before tuning anything, find out where the wait is.
- **HTTP API.** Every `200` carries a `Server-Timing` header. On a streamed response, its `ttfa` is our server's own time from receiving the request to sending the first audio byte. Your client's time to the first byte, minus `ttfa`, is the network, the connection setup and your client. When that difference is large, the connection and data zone habits below are the ones that help. See [Response headers](/tts/api-reference/http-api/overview#response-headers).
- **WebSocket API.** Time a turn from the `Generate` you send to the first `Audio` frame, and time opening a session, from connecting to `Begin`, separately. The first is what a turn waits for on an open session. The second is what a turn adds when it opens its own. When the text streams in from an LLM, also time the first audio from the moment the first sentence is complete: that is the wait [flushing each sentence](#send-text-as-it-is-written-and-flush-each-sentence) shortens.
The SDKs do not expose `Server-Timing`, so measure the HTTP split with a plain client. `httpx` ships with the Python SDK:
```python
import os
import time
import httpx
client = httpx.Client(headers={"Authorization": os.environ["ASSEMBLYAI_API_KEY"]})
body = {"text": "Hi, thanks for calling. How can I help you today?", "voice_id": "aai-kyla"}
for _ in range(5):
start = time.perf_counter()
with client.stream("POST", "https://tts.us.assemblyai.com/v1/speech", json=body) as response:
next(response.iter_bytes()) # the first audio bytes
client_ms = (time.perf_counter() - start) * 1000
server_timing = response.headers["server-timing"] # "open;dur=2.7, model;dur=40.4, ttfa;dur=43.1"
print(f"client {client_ms:.0f} ms server {server_timing}")
```
The first run also opens the connection; the rest run on the kept connection. The difference between the client's time and `ttfa` is what the habits below can remove.
## Use one WebSocket session per conversation
A conversation belongs on the [WebSocket API](/tts/connect-parameters), with one session held open for the whole call and each reply a request on it. Text can be sent a token at a time as an LLM writes it, the server starts speaking once it holds complete sentences, and `Cancel` stops the request in flight when the user interrupts. The rest of this page assumes that shape. [Choose an interface](/tts/getting-started/choose-an-interface) covers the decision itself, and when whole texts over HTTP or the OpenAI-compatible route fit better.
## Open the session before the first reply
Opening a session costs a TCP and a TLS handshake, the WebSocket upgrade, then our server's checks before `Begin`: the credential, the rate limit and the voice. A session opened per sentence pays all of it on every turn, and spends the [sessions-per-minute limit](/tts/limits#session-lifetime) on every turn as well. A turn on a session opened earlier pays none of it. Even close to the API, that setup delays the first audio of a turn that opens its own session, and the farther your client is from the API, the more the three round trips of a new WebSocket add.
Right after `Begin`, the server loads the session's voice. A voice the server has not used recently takes longer the first time, so a first turn sent right after `Begin` with such a voice waits a little longer for its first audio. Once the voice has loaded, a turn starts about as quickly as with a voice in recent use. A session opened ahead of the first reply gives the voice that time.
Open the session when the call starts or the page loads, not when the first reply is ready, and keep it open between turns. The server closes a session after two minutes without a frame in either direction, or after your shorter [`inactivity_timeout`](/tts/connect-parameters#inactivity-timeout), so send `KeepAlive` at least once per idle timeout while the caller is silent. Count the time your player spends on audio you already received as idle time too: audio arrives several times faster than it plays, so a long reply can leave the socket quiet while it is still playing. `KeepAlive` only restarts the idle timer, so sending it more often makes nothing faster, and a tight loop closes the session with `3010`. A session ends at `Begin.expires_at`, at most an hour after it opened. Open its replacement in a pause before then, send the following turns on it once its `Begin` arrives, and `Terminate` the old one, so no turn waits for a reconnect. One session per conversation is enough: an idle session still holds resources on our servers, so do not keep spares open in reserve.
```python
# Fragment. You supply call_is_open(), next_reply() and play().
import os
import assemblyai as aai
config = aai.TTSConfig(voice_id="aai-kyla", language="english")
# When the call starts, not when the first reply is ready.
session = aai.TTSSession(config, api_key=os.environ["ASSEMBLYAI_API_KEY"])
session.connect() # waits for Begin; the server then loads the voice before the first reply
while call_is_open():
reply = next_reply(timeout=60) # your agent; None after 60 s of silence
if reply is None:
session.keep_alive() # restarts the idle timer, nothing else
continue
for chunk in session.speak(reply):
play(chunk)
session.terminate()
```
```javascript
// Fragment. You supply callIsOpen(), nextReply() and play().
import { AssemblyAI } from "assemblyai";
const client = new AssemblyAI({ apiKey: process.env.ASSEMBLYAI_API_KEY });
// When the call starts, not when the first reply is ready.
const session = client.tts.session({ voiceId: "aai-kyla", language: "english" });
await session.connect(); // waits for Begin; the server then loads the voice before the first reply
while (callIsOpen()) {
const reply = await nextReply(60_000); // your agent; null after 60 s of silence
if (reply === null) {
session.keepAlive(); // restarts the idle timer, nothing else
continue;
}
for await (const chunk of session.speak(reply)) play(chunk);
}
await session.close();
```
The integrations do this for you. [LiveKit](/tts/livekit#latency) opens the session when you call `tts.prewarm()` at the start of the call, and [Pipecat](/tts/pipecat#latency) connects when the pipeline starts.
### Reuse one TLS context in Python
With the raw `websockets` package, `websockets.connect` builds a new TLS context for every connection unless you give it one. On Python linked against OpenSSL 3.0, such as the system Python of Ubuntu 24.04, building one takes noticeable CPU time, up to tens of milliseconds on a busy machine. Build one when your process starts and pass it to every connection. The SDKs and HTTP clients such as httpx build their context once, when the client is created, which is one more reason to keep one client per process.
```python
import ssl
import websockets
# Once per process. One context can be shared by any number of connections.
SSL_CONTEXT = ssl.create_default_context()
async def open_session(url, headers):
"""Open a session; the caller reads Begin and closes the connection."""
return await websockets.connect(url, additional_headers=headers, ssl=SSL_CONTEXT)
```
With a shared context, a turn that opens its own session no longer waits for a new context to be built before its first audio.
## Keep HTTP connections open
If you send whole replies over HTTP, the same idea applies to the connection. Build one HTTP client, or one OpenAI client, per process, and send every request through it. A new client starts with no open connection, and in Python it also builds a new TLS context.
The API keeps an idle connection open for an hour, but common clients close it much sooner: Python's httpx and httpx2, which the OpenAI Python SDK is built on, after 5 seconds, and Node's built-in `fetch` after 4. A voice agent whose replies are further apart than that opens a new connection, a TCP and a TLS handshake, before nearly every reply. Raise the client's idle limit above your longest pause between replies; 2 minutes covers most conversations.
```python
import httpx
# One client per process: httpx's own pool sizes, with idle connections kept
# for 2 minutes instead of 5 seconds.
client = httpx.Client(
limits=httpx.Limits(max_connections=100, max_keepalive_connections=20, keepalive_expiry=120),
timeout=httpx.Timeout(5.0, read=30.0),
)
```
```javascript
import { Agent, fetch } from "undici";
// One agent per process: idle connections kept for 2 minutes instead of 4 seconds.
// Pass undici's own fetch with its Agent; npm install undici.
const agent = new Agent({ keepAliveTimeout: 120_000 });
const response = await fetch("https://tts.us.assemblyai.com/v1/speech", {
method: "POST",
dispatcher: agent,
headers: { Authorization: process.env.ASSEMBLYAI_API_KEY, "Content-Type": "application/json" },
body: JSON.stringify({ text: "Hello from AssemblyAI." }),
});
```
For the OpenAI SDKs, Pipecat and LiveKit, see [OpenAI compatibility](/tts/openai-compatibility#keep-connections-between-replies).
The first request through a client still opens a connection. Open it while nobody is waiting, for example when a call starts, with any cheap request through the same client. `HEAD /v1/openai/models` takes no credential and synthesizes nothing:
```python
client.head("https://tts.us.assemblyai.com/v1/openai/models")
```
The connection then stays open for the client's idle limit, so open it shortly before the first reply rather than at application startup, unless you raised the limit as above.
Every reconnect costs two round trips before the request can leave, with any client: a few milliseconds close to the API, and more the farther your client is from it.
## Send text as it is written, and flush each sentence
Send each piece of text with `Generate` as soon as you have it, and keep the space at the edge of each piece, since pieces are joined exactly as sent. The first audio of a reply cannot come before its first sentence has arrived, so text held back on the client delays it by as much.
Send `Flush` at the end of each sentence as it completes, starting with the first. A `Flush` starts that sentence's audio at once, so the reply starts right after its first sentence is complete. Without one, the server waits until it holds enough complete sentences, so a short first sentence such as `Sure, I can help with that.` waits for the sentence after it, and a reply sent in one piece after the LLM finishes waits for the whole reply. At the pace an LLM writes, either can add close to a second before the reply starts.
Never flush per token, or in the middle of a sentence. A request that ends mid-sentence is synthesized on its own, which produces choppy prosody, and a per-token loop exceeds the control-frame limit and closes the session. Always flush when a reply is complete: without it, the end of the reply is held until more text arrives, and a short reply is never spoken at all. With the SDKs, a `speak()` per sentence, or a `flush()` after the last `generate()` of each sentence, does the same. See [Generate](/tts/message-frames#generate) and [Limits](/tts/limits).
Keep the first sentence of a reply short. With a flush per sentence, the audio starts once the first sentence is complete, so a short opening starts it sooner than a long one with the same content.
## Stream raw PCM in the format your player needs
A streamed response sends its first byte with the first audio. A buffered one, `"stream": false` on the HTTP API, sends nothing until the whole clip is ready, which is right for a file and wrong for a conversation. MP3 starts every streamed response with the encoder's delay, about 1,100 samples of silence, which is about 46 ms at 24 kHz, 69 ms at 16 kHz and 138 ms at 8 kHz, and its first byte also comes a little later than PCM's. For a voice agent, stream `pcm_s16le`, which is `pcm` on the OpenAI-compatible route, where MP3 is the default.
The service's native output is 24 kHz 16-bit PCM. Every other rate or codec is converted on the server before it reaches you, which adds a few milliseconds at most, so set the format your playback path needs explicitly rather than converting it yourself. 16 kHz `pcm_s16le` is the common choice when the audio feeds a pipeline that already runs at 16 kHz, and 8 kHz `pcm_mulaw` or `pcm_alaw` matches a carrier media stream directly. A rate above 24 kHz is resampled up from the native output, so it carries no more detail and only adds bytes to send; ask for 44.1 or 48 kHz only when your output device needs it. Read from the socket continuously and buffer the audio locally. Your player, not the socket, should be what waits. See [Audio formats](/tts/connect-parameters#audio-formats) on WebSocket API basics, and the [HTTP API's](/tts/api-reference/http-api/overview#audio-formats) WAV and MP3 containers.
## Use the nearest data zone
`tts.us.assemblyai.com` runs in us-east-1 (N. Virginia) and `tts.eu.assemblyai.com` in eu-north-1 (Stockholm). Choose the zone your data residency requires first, and the zone that holds your [cloned voices](/tts/cloned-voices#data-zones), since a voice is only usable in the zone it was created in. With no such requirement, choose the zone nearer to the machine that calls the API. If the same agent also streams the caller's audio to Streaming Speech-to-Text, use the same data zone for both and run the agent near it, since every turn makes round trips to both. See [Cloud endpoints and data residency](/tts/endpoints-and-data-zones).
Every request costs at least one network round trip to the zone, and a new connection costs more before the request can leave: two round trips for HTTPS (TCP, then TLS 1.3) and three for a WebSocket (TCP, TLS and the upgrade). A round trip takes longer the farther your client is from its zone, so the farther it is, the more the connection habits above are worth.
## In a browser
A page opens each session with a [temporary token](/tts/authenticate-with-a-temporary-token), so a connection first waits for the page to ask your server and for your server to mint the token, which is a request of its own to our API. A token minted ahead of time takes that wait off the session's open.
- **Keep a spare token.** Mint it on an early hint that a session will be needed, such as the pointer entering the text box, and only when the page will have to open a session, such as after the idle session closed. While a session is open, a spare goes unused and only sits in the page as a live credential. Drop it well before its `expires_in_seconds` runs out, and give every connection a token of its own.
- **Reopen on intent.** A page that lets an idle session close and reopens it when the user presses Speak makes that click wait for the token, the handshakes and `Begin`. Reopen when the user focuses the text box instead, while they type, and the click finds the session open. A click that comes before the reopened session's `Begin` should keep its text and speak it when `Begin` arrives.
## Write the text the way it should be read
Before synthesis, the server rewrites the forms that identify themselves into the words they are read as, and drops what is written but never said, in every language. For most sentences that is what you want: write currencies, phone numbers, addresses and dates the way they are normally written, and they are read the way they are normally said.
- **Read in their spoken form:** amounts with a currency sign, dates written with their year, phone numbers, addresses, units, email addresses and URLs. `$25.50` is read as twenty five dollars and fifty cents, and `(415) 555-0132` in groups of digits. An English numeric date such as `03/04/2026` is read month first, as March fourth.
- **Also read out in English:** common written shorthand and technical strings. `p. 12` is read as page 12, `w/` as with and `press *` as press star, and a file path, a MAC address or a `0x` hex number symbol by symbol.
- **Dropped:** emoji, Markdown and HTML markup, footnote marks such as the `[3]` in `as the study showed[3]`, and decorative symbols.
- **Left to the model:** anything else, such as a number with no format that says what it is, like `4321` or `1,234`. The model reads it on its own.
Where the reading matters, make it explicit.
- **Use full sentences with punctuation.** The model takes pace and tone from it. Text with no punctuation may not be read in the sentences you expect, and a question mark, an exclamation mark or a comma each change the delivery.
- **Spell out a reading the model could get wrong.** Put dashes between digits to read a number digit by digit, as in a card number, and write it with commas to read it as a whole number. Put dashes between the letters of an uncommon acronym. Write out a date or an amount in words when the written form is ambiguous, such as a numeric date meant day first or one written without its year.
- **Remove anything that should not be spoken.** Stage directions are read aloud: square brackets become parentheses, so `[pause]` is read as the word pause, not as a pause instruction.
- **Send complete sentences per request.** A request that ends mid-sentence is spoken on its own, without the context of the sentence it belongs to, which is the same reason not to flush per token.
See the [Prompting guide](/tts/prompting-guide) for the full normalization table and the punctuation the model responds to.
## Set the language of the text
`language` decides the language the session speaks and the model it runs on. Set it to the language of the text you send: the API does not detect it. When you leave it out, the session speaks the language its voice was recorded in, which is English for a session that names no voice. `assemblyai-tts-1` speaks every voice in all six languages, so any voice can read your text once `language` matches it, though results are usually worse than in the voice's own language: for the best results, choose a voice whose own language is the language of your text. Text in another language than the session's comes out mispronounced. Read `configuration.voice_id` and `configuration.language` back from `Begin` to confirm what the session resolved, defaults included. See [Voices](/tts/voices#voice-and-language).
## Related
- [Message frames](/tts/message-frames): when the server starts speaking a `Generate`, and what `Flush`, `Cancel` and `KeepAlive` do.
- [Limits](/tts/limits): text, session and rate limits, and how each one reports a breach.
- [Prompting guide](/tts/prompting-guide): pacing, tone and normalization.
- [WebSocket API basics](/tts/connect-parameters#audio-formats): the encodings and sample rates, and playing raw PCM from `Begin`'s sample rate.
- [OpenAI compatibility](/tts/openai-compatibility#keep-connections-between-replies): keeping connections with the OpenAI SDKs, Pipecat and LiveKit.
- [HTTP API](/tts/api-reference/http-api/overview): streamed and buffered responses, audio formats and `Server-Timing`.
---
# Choose an interface
URL: https://www.assemblyai.com/docs/tts/getting-started/choose-an-interface
Source: docs/tts/getting-started/choose-an-interface.mdx
Navigation: Text to Speech > Getting started
Description: The SDKs, the HTTP API, the WebSocket API, the OpenAI-compatible route and the LiveKit and Pipecat plugins, and which one fits how your text arrives.
Streaming TTS speaks with one set of voices and models, bills the same characters and runs on the same hosts whichever way you call it. What differs is the shape of the conversation with the server. The HTTP API takes a whole text in one request and answers with the audio. The WebSocket API keeps one connection open for a whole conversation and speaks text as it arrives. The SDKs, the OpenAI-compatible route and the LiveKit and Pipecat plugins are ways of calling one of those two. Pick by how your text arrives and by what you already run.
| Interface | Protocol | Use it when |
|-----------|----------|-------------|
| [Python and JavaScript SDKs](/tts/getting-started/quickstart) | Both | You write Python or Node.js and want the requests, the audio framing and the errors handled for you |
| [HTTP API](/tts/api-reference/http-api/overview) | HTTP | You have a whole sentence or text in hand, want a file, or call from a language without an SDK |
| [WebSocket API](/tts/connect-parameters) | WebSocket | Text arrives word by word, and one conversation should run on one connection with barge-in |
| [OpenAI-compatible route](/tts/openai-compatibility) | HTTP | Your code or framework already speaks the OpenAI speech API |
| [LiveKit plugin](/tts/livekit) | WebSocket | Your agent runs on LiveKit Agents |
| [Pipecat service](/tts/pipecat) | WebSocket | Your agent runs on Pipecat |
## Start with an SDK
The Python and JavaScript SDKs cover both protocols. `synthesize()` sends one text over HTTP and returns the whole audio, `stream()` sends one text over HTTP and yields the audio as it is made, and a session object speaks a conversation over one WebSocket connection with flush, cancel and the audio of each request in order. They also raise a refused request or a rejected session as a typed error, so you never parse an error body or frame yourself. The [Quickstart](/tts/getting-started/quickstart) shows all three, and every feature page shows its SDK form first.
Reach for the raw protocol when you work in another language, when you run in a browser or on a device without the SDK, or when you need something the SDK does not expose yet. The quickstart's cURL tab and its [direct WebSocket program](/tts/getting-started/quickstart#using-the-websocket-api-directly) are the starting points.
## Whole texts over HTTP
The [HTTP API](/tts/api-reference/http-api/overview) turns one text into speech in one request. The audio is the response body, streamed while it is synthesized by default or as a whole file on request, and it comes as raw PCM, G.711, WAV or MP3. A second route answers the audio together with the time at which each word is spoken. Every request stands alone, a failure before the audio starts is an HTTP status with a JSON body, and nothing has to be kept alive between requests.
This fits a voice agent that sends one request per sentence, a server that renders prompts or notifications to files, a script, and a page that calls the API with a [temporary token](/tts/authenticate-with-a-temporary-token). Keep the connection open between requests so only the first one pays for the handshake. Each request has its own character limit, which [Limits](/tts/limits) lists with the others.
## Conversations over WebSocket
The [WebSocket API](/tts/connect-parameters) holds one connection for a whole conversation. Each sentence is a request on it, text can be sent a token at a time as an LLM writes it, the server starts speaking once it holds complete sentences, and a cancel stops the request in flight for barge-in. Word boundaries arrive with the audio. The audio is raw codec bytes, PCM or G.711, with no file container, which is what a player that starts on the first bytes wants.
This fits anything that talks back and forth: a voice agent, a live assistant, a dictation reader. Opening a connection per sentence pays the handshake every time and runs into the limit on new sessions, so one session per conversation is the shape to aim for. [Message frames](/tts/message-frames) documents the protocol and [Audio formats](/tts/connect-parameters#audio-formats) the bytes.
## Already on the OpenAI speech API
If your code or framework calls the OpenAI speech API, point it at the [OpenAI-compatible route](/tts/openai-compatibility) with your AssemblyAI API key and it speaks with AssemblyAI voices without other changes. The OpenAI SDKs work this way, and so do the OpenAI plugins of LiveKit and Pipecat, though both frameworks also have native AssemblyAI plugins that run on the WebSocket API. It is the HTTP API with OpenAI's request fields and error body. It always streams, in MP3 unless you ask for PCM, WAV or G.711, so the native routes remain the place for buffered responses, the MP3 bit rate and word timings.
## Where to next
- [Quickstart](/tts/getting-started/quickstart): first audio with an SDK or cURL, then a conversation over one session.
- [HTTP API basics](/tts/api-reference/http-api/overview): hosts, authentication, response framing, audio formats, errors and limits of the HTTP routes.
- [WebSocket API basics](/tts/connect-parameters), [Message frames](/tts/message-frames) and the [WebSocket API reference](/tts/api-reference/websocket-api): the WebSocket API in full.
- [OpenAI compatibility](/tts/openai-compatibility): base URLs, models, voices and framework setups.
- [LiveKit](/tts/livekit) and [Pipecat](/tts/pipecat): the AssemblyAI plugins for the two frameworks.
---
# Voices
URL: https://www.assemblyai.com/docs/tts/voices
Source: docs/tts/voices.mdx
Navigation: Text to Speech > Features
Description: Name a voice with voice_id in the SDKs and on the wire, where to find voices, the ones to start with, and how voice and language fit together.
Every voice is named by its voice ID: the `voice_id` parameter on a WebSocket session and on an HTTP request, `voice_id` in the Python SDK's `TTSConfig`, and `voiceId` on a JavaScript SDK session (`voice_id` in its HTTP requests, which take the API's field names). It takes one of these:
- The library slug of an AssemblyAI voice, such as `aai-kyla`.
- A voice's ID, a UUID.
- The slug of one of your account's voices, such as a voice you [cloned](/tts/cloned-voices) in the AssemblyAI dashboard.
Leave `voice_id` out and you get the [default voice](#the-default-voice) of the session's language: `aai-kyla` in English.
```python
# You supply play().
import assemblyai as aai
config = aai.TTSConfig(voice_id="aai-kyla")
with aai.TTSSession(config) as session: # the WebSocket API
for chunk in session.speak("Hi, thanks for calling."):
play(chunk)
print(session.begin.configuration["voice_id"])
response = aai.TextToSpeech(config=config).synthesize("Hi, thanks for calling.") # the HTTP API
print(response.voice_id)
```
The SDK reads your API key from `ASSEMBLYAI_API_KEY`, or from `aai.settings.api_key`.
```javascript
// You supply play().
import { AssemblyAI } from "assemblyai";
const client = new AssemblyAI({ apiKey: process.env.ASSEMBLYAI_API_KEY });
const session = client.tts.session({ voiceId: "aai-kyla" }); // the WebSocket API
const begin = await session.connect();
for await (const chunk of session.speak("Hi, thanks for calling.")) play(chunk);
await session.close();
console.log(begin.configuration.voice_id);
const response = await client.tts.synthesize({ text: "Hi, thanks for calling.", voice_id: "aai-kyla" }); // the HTTP API
console.log(response.voiceId);
```
On the WebSocket API, `voice_id` is a query parameter of the URL you open; on the HTTP API, a field of the request body:
```text
wss://tts.us.assemblyai.com/v1/ws?voice_id=aai-kyla&sample_rate=24000&encoding=pcm_s16le
```
```bash
curl -sS --fail-with-body https://tts.us.assemblyai.com/v1/speech \
-H "Authorization: $ASSEMBLYAI_API_KEY" \
-d '{"text": "Hi, thanks for calling.", "voice_id": "aai-kyla", "container": "wav"}' \
-o kyla.wav
```
The session or request echoes the voice it speaks with: `Begin.configuration.voice_id` on a session, which the SDKs expose on the `Begin` event, and the `X-TTS-Voice-Id` header on an HTTP response, which the SDKs expose on the result. It is the value you sent, with a UUID in lowercase, or the default voice's slug, such as `aai-kyla`, when you sent none. On a session, `Begin.configuration.resolved_voice.voice_id` gives the voice's UUID, also when you named it by slug.
`voice`, which named a voice by a bare name such as `voice=kyla`, is retired. A session or request that still sends it, with any value, is refused before any audio: `3006` on the WebSocket API and `422` on the HTTP API, with `error_type` `invalid_parameter` and `param` `voice`. Send the voice's library slug or ID in `voice_id` instead.
## AssemblyAI voices
The [voice catalog](https://www.assemblyai.com/dashboard/text-to-speech/voices) in the AssemblyAI dashboard holds thousands of voices across the six languages the model speaks. Browse it there, listen to each voice, and copy its `voice_id` with **Copy voice ID**; the text-to-speech playground has the same action, which copies the `voice_id` its session sends. AssemblyAI's voices carry slugs that begin with `aai-`. Each voice was recorded in one language, which the catalog shows, and speaks that language when you leave `language` out; see [Voice and language](#voice-and-language).
These are good voices to start with. The examples on these pages use Kyla:
| Voice | `voice_id` | Language | Description |
|-------|------------|-------------|-------------|
| Aaliyah | `aai-aaliyah` | English, American | Young woman speaking American English with a fairly high-pitched voice |
| Abdiel | `aai-abdiel` | English | An older man speaking English with a breathy voice |
| Annabelle | `aai-annabelle` | English, American | A young woman with a high-pitched voice speaking American English |
| April | `aai-april` | English, American | A young woman speaking American English at a fast pace |
| Audrey | `aai-audrey` | English, American | A middle-aged woman with an even, steady intonation speaks American English |
| Buford | `aai-buford` | English, British | An older man speaking British English with an energetic delivery |
| Carrie | `aai-carrie` | English, British | A middle-aged woman with a soft-spoken delivery in British English |
| Cloyd | `aai-cloyd` | English, British | An older man with a high-pitched, fast-paced, energetic delivery in British English |
| Dorsey | `aai-dorsey` | English | An older man's voice with a raspy, textured, nasal, and breathy quality |
| Ernest | `aai-ernest` | English, American | A middle-aged man speaking fast-paced American English with a high-pitched voice |
| Ewald | `aai-ewald` | English, American | A middle-aged man speaking American English at a fairly fast pace |
| Gunnar | `aai-gunnar` | English | An older man speaking English at a fairly slow pace |
| Hailey | `aai-hailey` | English, American | A young woman speaking American English at a fairly fast pace |
| Kyla | `aai-kyla` | English, American | A young woman speaking American English at a fast pace |
| Lilla | `aai-lilla` | English, British | A middle-aged woman with energetic delivery and a nasal, expressive intonation in British English |
| Nancy | `aai-nancy` | English, British | A middle-aged woman speaking British English with a fairly even tone |
| Percival | `aai-percival` | English, British | An older man with energetic delivery and expressive intonation, speaking British English in a breathy manner |
| Peter | `aai-peter` | English, American | A middle-aged man with a soft-spoken delivery in American English |
| Sara | `aai-sara` | English, American | A young woman speaking American English with an even, steady intonation |
| Steven | `aai-steven` | English, Australian | A middle-aged man speaking Australian English with a fairly even tone |
| Titus | `aai-titus` | English, American | A young man with a deep, low-pitched voice speaking American English |
| Troy | `aai-troy` | English, Scottish | A young man with a soft-spoken, deep, low-pitched voice and expressive intonation in Scottish English |
| Yasmin | `aai-yasmin` | English, British | A middle-aged woman with expressive intonation speaks British English |
| Antonio Luis | `aai-antonioluis` | Spanish, Mexican | A deep, low-pitched voice of a middle-aged man speaking Mexican Spanish |
| Constanza | `aai-constanza` | Spanish, Mexican | A young woman speaking Mexican Spanish with an even, steady intonation |
| Malvina | `aai-malvina` | Spanish, Mexican | A soft-spoken middle-aged woman speaking Mexican Spanish |
| Oriol | `aai-oriol` | Spanish, Mexican | A middle-aged man speaking Mexican Spanish at a fairly fast pace |
| Günther | `aai-gunther` | German | A middle-aged man speaking German with expressive intonation and a soft-spoken delivery |
| Jörg | `aai-jorg` | German | A middle-aged man speaking German with a fairly even tone |
| Magdalena | `aai-magdalena` | German | A young woman with a deep, low-pitched voice speaking German |
| Giada | `aai-giada` | Italian | A young woman speaking Italian with a clear, bright tone and an energetic delivery |
| Donatien | `aai-donatien` | French | A middle-aged man with a high-pitched, soft-spoken voice and expressive intonation |
| Benício | `aai-benicio` | Portuguese | A young man speaking Portuguese with a balanced, natural, easy-to-listen-to voice |
| Nelson | `aai-nelson` | Portuguese, Brazilian | An older man speaking Brazilian Portuguese with a deep, low-pitched, raspy voice |
### The default voice
A session or request that leaves `voice_id` out speaks with the default voice of its language, a catalog voice recorded in that language:
| Language | Default voice | `voice_id` |
|----------|---------------|------------|
| English | Kyla | `aai-kyla` |
| Spanish | Antonio Luis | `aai-antonioluis` |
| German | Magdalena | `aai-magdalena` |
| Italian | Giada | `aai-giada` |
| Portuguese | Nelson | `aai-nelson` |
| French | Donatien | `aai-donatien` |
With `language` left out as well, the session speaks English with Kyla. `Begin.configuration.voice_id` and the `X-TTS-Voice-Id` header name the default voice that spoke. The defaults may change, so name a voice with `voice_id` to keep the same one.
### The first session with a voice
The first session with a voice that has not been used recently can wait a little longer for its first audio when its text arrives right after `Begin`, and not at all when the voice is ready by the time the text arrives. Open the session when the call starts, before your first reply is ready, and keep it open for the conversation; see [Optimizing accuracy and latency](/tts/getting-started/optimizing-accuracy-and-latency).
## Voice and language
`language` is the language the session speaks. Set it to the language of the text you send. When you leave it out, a catalog voice or a [cloned voice](/tts/cloned-voices#language-and-model) speaks its own language, the one it was recorded in, and a session that names no voice speaks English. `language` also decides which model the session runs on: the default model of that language in the region, `assemblyai-tts-1` for all six languages.
`assemblyai-tts-1` speaks every voice in all six languages: set `language` and the voice speaks it, whatever language the voice was recorded in. To pick a voice recorded in the language of your text, such as Antonio Luis for Mexican Spanish, look at the language the catalog shows for it.
A voice usually sounds best in its own language. Speech in another language
is usually of lower quality, so for the best results choose a voice whose
own language is the session's language.
```python
# Fragment: selected_voice_id comes from your voice picker; text_language is the
# language of the text you will send.
config = aai.TTSConfig(voice_id=selected_voice_id, language=text_language)
```
```javascript
// Fragment: selectedVoiceId comes from your voice picker; textLanguage is the
// language of the text you will send.
const session = client.tts.session({ voiceId: selectedVoiceId, language: textLanguage });
const response = await client.tts.synthesize({ text, voice_id: selectedVoiceId, language: textLanguage });
```
```javascript
// Fragment: selectedVoiceId comes from your voice picker, and textLanguage is
// the language of the text you will send.
const params = new URLSearchParams({
voice_id: selectedVoiceId,
language: textLanguage,
sample_rate: "24000",
encoding: "pcm_s16le",
});
```
`Begin.configuration.language` names the language the session speaks, defaults included, and `Begin.configuration.resolved_voice.language` the language the voice was recorded in. On the HTTP API, the `X-TTS-Language` header names the language of the request.
`language` takes the full English name of the language, such as `spanish`, not a code such as `es`, in any letter case. This differs from the STT APIs, which take short language codes such as `es` or `en_us`. The [OpenAI-compatible route](/tts/openai-compatibility#extensions) and the [LiveKit plugin](/tts/livekit#set-the-language) accept an ISO 639-1 code such as `es` as well.
The API does not detect language. Set `language` to the language of your
text: without it the session speaks the voice's own language, or English
when you name no voice, whatever language the text is in.
## Languages
`language` takes one of these values:
| Language | `language` value |
|----------|------------------|
| English | `english` |
| Spanish | `spanish` |
| German | `german` |
| Italian | `italian` |
| Portuguese | `portuguese` |
| French | `french` |
A session speaks a language only where a model of the region speaks it. Both data zones serve all six languages on one model, `assemblyai-tts-1`, the default model of each of them, and `Begin.configuration.model` names the model a session runs on. A `language` outside this table closes the session with `3006` before `Begin`, with `error_type` `invalid_parameter` and `param` `language`, and the text lists the supported values. A supported language that no model of the region speaks closes with `3006`, `error_type` `invalid_request`, `param` `language` and the text `No voice model serves language in this region`.
## What a wrong voice looks like
The server checks the voice when the session opens. A voice it cannot use never gets `Begin`: an `Error` frame arrives in its place, and the socket closes with the frame's `error_code`. The Python SDK raises it as `TTSError` from the `with` block, the JavaScript SDK rejects `connect()` with a `TextToSpeechError`; both carry the frame's `error_type` as `error_code` (`errorCode`) and its `param`. On the HTTP API the same refusals are `422` answers with the same codes, which the SDKs raise from `synthesize()` and `stream()`.
- **No voice with this `voice_id`.** A `voice_id` that names no voice your account can use on this host, such as a cloned voice that was [deleted](/tts/cloned-voices#delete-a-voice), closes the session with `3012`, `error_type` `voice_not_found` and `param` `voice_id`. A bare name such as `voice_id=kyla` is not a library slug: it is looked up among the slugs of your own voices, and unless one of your voices has that slug, it is not found either. Send the library slug, `aai-kyla`.
- **An AssemblyAI voice the session's model does not speak.** An AssemblyAI voice that the session's model does not speak in the session's language closes the session with `3006`, `error_type` `invalid_request` and `param` `voice_id`, and the message lists the voices available on that connection by their slugs.
With a `voice_id` that names no voice (`voice_id=aai-nonexistent`):
```json
{
"type": "Error",
"error_code": 3012,
"error": "The requested voice was not found. It may not exist, may have been deleted, or may be another account's private voice.",
"error_type": "voice_not_found",
"retryable": false,
"param": "voice_id",
"retry_after_seconds": null
}
```
Fix the voice before you reconnect; retrying unchanged fails identically. Keep handling a `3006` whose `param` is `voice_id` after `Begin` as well: in rare cases the refusal of an AssemblyAI voice the model does not speak arrives right after `Begin`, with the same text, before any audio. Read `configuration.voice_id` and `configuration.language` from `Begin` to confirm what the session resolved, defaults included. See [Error handling](/tts/error-handling).
## Related
- [Voice cloning](/tts/cloned-voices): a voice you clone in the AssemblyAI dashboard, named by its `voice_id`.
- [WebSocket API basics](/tts/connect-parameters): the connect sequence and the rules for the rest of the query string.
- [Error handling](/tts/error-handling): what to do with `3006`, `3012` and the other close codes.
- [Optimizing accuracy and latency](/tts/getting-started/optimizing-accuracy-and-latency): open the session before the first reply.
- [Quickstart](/tts/getting-started/quickstart): a full session using Kyla.
---
# Voice cloning
URL: https://www.assemblyai.com/docs/tts/cloned-voices
Source: docs/tts/cloned-voices.mdx
Navigation: Text to Speech > Features
Description: What a cloned voice is, how to create one in the AssemblyAI dashboard, how to speak with it or with a library voice from the SDKs, the HTTP API and the OpenAI-compatible route, how to keep its consent current, and how to delete it in the dashboard or with the voices API.
Voice cloning makes a voice from a short recording of one person speaking. A cloned voice belongs to the account that created it: a new voice is private to that account, and only that account can renew its consent or delete it. Cloning needs the speaker's consent, and a cloned voice can be used for 60 days after its consent was last given, unless you renew the consent before then; see [Consent period](#consent-period).
You create and manage clones in the [voice catalog](https://www.assemblyai.com/dashboard/text-to-speech/voices) of the AssemblyAI dashboard, and the API speaks with them like with any other voice: pass the voice's slug in `voice_id`. The frames, audio formats and limits are those of an [AssemblyAI voice](/tts/voices).
## Create a voice
In the voice catalog, choose **Create voice clone**, then record or upload a sample:
- **One person, speaking the voice's language.** Pick the language in the dialog and have the speaker read in it: a speaker reading English into a voice set to French is cloned on the wrong sounds.
- **Ten to thirty seconds of speech.** The dashboard uses up to thirty seconds of a recording under two minutes long, and shows when a recording is long enough. Read the script it offers, calm or excited, or your own text at a natural pace.
- **Consent.** You attest that the speaker consented to the clone, and your account must have accepted the biometric data terms on the dashboard's [data controls](https://www.assemblyai.com/dashboard/settings/data-controls) page.
The new voice gets a slug, which you can choose when you create it and not after, and an ID, a UUID. Name the voice by its slug in `voice_id`, in lowercase; its ID works too.
### Get the best clone
The quality of a clone depends heavily on the recording you supply: the clone reproduces the voice, and also how it was recorded and how it was spoken. For the highest quality clones:
- **Use a good microphone, close to the mouth.** The closer the microphone, the more of the voice it picks up. A headset or earbuds, such as AirPods, work well.
- **Record somewhere quiet, with little echo.** Background noise and room echo come through as artifacts in the generated speech. A small room with soft furnishings echoes less than a large, bare one.
- **Upload a WAV file.** An uncompressed file keeps detail that MP3 and other compressed formats lose.
- **Speak in the tone and pitch you want to hear.** The voice speaks the way the recording does, so record calm for a calm voice and upbeat for an upbeat one.
- **Keep the quirks you want in the voice.** Raspiness, filler words and hesitations, and the way the speaker draws out or shapes words carry into the generated speech. If you read a script, read it the way you would say it.
Hear the difference in two clones of the same speaker, one recorded on a webcam microphone and one on AirPods. The AirPods clone keeps more of the speaker's tone and quirks.
## Speak with it
Name the voice in `voice_id` and leave out `language`. The programs below read your API key and the voice's slug from the environment, open a session with the voice, print what `Begin` resolved and speak one sentence; the cURL tab does the same as one HTTP request, and the OpenAI SDK tab through the [OpenAI-compatible route](/tts/openai-compatibility). Set the two variables first:
```bash
export ASSEMBLYAI_API_KEY=
export ASSEMBLYAI_VOICE_ID=
```
```python
import os
import assemblyai as aai
config = aai.TTSConfig(voice_id=os.environ["ASSEMBLYAI_VOICE_ID"])
with aai.TTSSession(config, api_key=os.environ["ASSEMBLYAI_API_KEY"]) as session:
c = session.begin.configuration
print(f"voice_id {c['voice_id']}, resolved {c['resolved_voice']['voice_id']}")
print(f"language {c['language']}, model {c['model']}")
print(f"voice expires at {c['voice_expires_at']}")
with open("cloned.pcm", "wb") as out:
for chunk in session.speak("Hi, thanks for calling. How can I help you today?"):
out.write(chunk) # raw 24 kHz 16-bit mono PCM
```
A voice the session cannot use raises `aai.TTSError` from the `with`, with the refusal's `error_type` as `error_code`, such as `voice_not_found`, and the close code in `body["close_code"]`.
```javascript
import { createWriteStream } from "node:fs";
import { AssemblyAI } from "assemblyai";
const client = new AssemblyAI({ apiKey: process.env.ASSEMBLYAI_API_KEY });
const session = client.tts.session({ voiceId: process.env.ASSEMBLYAI_VOICE_ID });
const { configuration: c } = await session.connect();
console.log(`voice_id ${c.voice_id}, resolved ${c.resolved_voice.voice_id}`);
console.log(`language ${c.language}, model ${c.model}`);
console.log(`voice expires at ${c.voice_expires_at}`);
const out = createWriteStream("cloned.pcm");
for await (const chunk of session.speak("Hi, thanks for calling. How can I help you today?")) {
out.write(chunk); // raw 24 kHz 16-bit mono PCM
}
out.end();
await session.close();
```
A voice the session cannot use rejects `connect()` with a `TextToSpeechError`, with the refusal's `error_type` as `errorCode`, such as `voice_not_found`, and the close code in `closeCode`.
The [HTTP API](/tts/api-reference/http-api/overview) takes the voice in the body field `voice_id`:
```bash
curl -sS --fail-with-body -D headers.txt https://tts.us.assemblyai.com/v1/speech \
-H "Authorization: $ASSEMBLYAI_API_KEY" \
-d '{"text": "Hi, thanks for calling. How can I help you today?", "voice_id": "'"$ASSEMBLYAI_VOICE_ID"'", "container": "wav"}' \
-o cloned.wav
```
Among the headers saved in `headers.txt`, `x-tts-voice-id` is the `voice_id` you sent, `x-tts-language` the language spoken and `x-tts-voice-expires-at` the voice's expiry; the SDKs' `synthesize()` and `stream()` expose them on the result. On the WebSocket URL, the voice is the query parameter `voice_id=`.
The OpenAI-compatible route takes the voice in OpenAI's field `voice`, as an object that holds its slug:
```python
import os
from openai import OpenAI
client = OpenAI(api_key=os.environ["ASSEMBLYAI_API_KEY"], base_url="https://tts.us.assemblyai.com/v1/openai")
with client.audio.speech.with_streaming_response.create(
model="assemblyai-tts-1",
voice={"id": os.environ["ASSEMBLYAI_VOICE_ID"]},
input="Hi, thanks for calling. How can I help you today?",
response_format="pcm",
) as response:
print(f"voice_id {response.headers['x-tts-voice-id']}, language {response.headers['x-tts-language']}")
response.stream_to_file("cloned.pcm") # raw 24 kHz 16-bit mono PCM
```
Send the slug inside the object, as above: the slug of one of your voices in a plain string is refused. `model="assemblyai-tts-1"` speaks the voice's own language unless the `language` extension names another, and a model named for a language, such as `assemblyai-tts-es`, speaks that language. Either way the voice can speak a language other than its own, though results are usually worse than in the voice's own language; see [Language and model](#language-and-model) and [OpenAI compatibility](/tts/openai-compatibility#cloned-voices).
For an English voice, the output of the SDK programs looks like this:
```text
voice_id support-voice, resolved 01a139bf-78a0-7d18-9a4e-6c0f2b8d1e37
language english, model assemblyai-tts-1
voice expires at 1797154212
```
`Begin.configuration.voice_id` is the value you sent, and `resolved_voice.voice_id` is the voice's UUID, also when you sent its slug. `voice_expires_at` is when the voice's [consent period](#consent-period) ends. The other keys a cloned voice adds to `Begin` are in [Message frames](/tts/message-frames#begin).
The same programs speak with an AssemblyAI library voice: set `ASSEMBLYAI_VOICE_ID` to its slug, such as `aai-kyla`. A library voice does not expire, so `voice_expires_at` is `None` in Python and `null` in JavaScript, and the HTTP response carries no `x-tts-voice-expires-at` header. See [Voices](/tts/voices).
### Language and model
A cloned voice speaks the language of its recording when you leave `language` out: `Begin.configuration.language` shows it, and the default of `english` does not apply. Set `language` to another of the [six languages](/tts/voices#languages) and the session speaks that language with the voice: `assemblyai-tts-1` speaks every voice in all six. `Begin.configuration.resolved_voice.language` keeps the language of the recording.
A voice usually sounds best in its own language. Speech in another language
is usually of lower quality, so for the best results choose a voice whose
own language is the session's language.
Leave `model_id` out as well: the session runs on the default model of its language, `assemblyai-tts-1` today, and `Begin.configuration.model` names it. A voice is prepared for each model it speaks on, which the errors call a pairing; see [Errors](#errors) for a voice that is not ready yet.
### Data zones
A cloned voice belongs to the [data zone](/tts/endpoints-and-data-zones) it was created in, and only that zone's host can use it: a voice of the US data zone on `tts.us.assemblyai.com`, a voice of the EU data zone on `tts.eu.assemblyai.com`. On the other zone's host the voice is not found. Set the SDK's host accordingly: `aai.settings.tts_base_url` in Python, `ttsBaseUrl` on the JavaScript client. Every session with the voice goes to that zone's host, so create the voice in the zone nearest the people who will hear it: the network distance to the host adds to the time to first audio of every reply.
### First audio
A cloned voice is loaded when a session with it opens, so the first session with a voice that has not been used recently waits a little longer for its first audio when its text arrives right after `Begin`. Open the session when the call starts and keep it open for the conversation, so the voice is ready before your first reply; see [The first session with a voice](/tts/voices#the-first-session-with-a-voice).
## Consent period
A cloned voice can be used for 60 days after its consent was last given, until `voice_expires_at`, a Unix timestamp in seconds that `Begin`, the HTTP response headers and the SDK results carry. After that, sessions and requests with the voice are refused with `voice_expired`. An expired voice cannot be renewed or used again: create the voice again from a new consented recording, which gives it a new ID. Renew a voice you want to keep before it expires.
### Renew consent
The voice's owner renews its biometric consent, which starts the 60 days again, by setting `extend_biometric_consent` to `true` on a session or request that uses the voice. Setting it attests renewed consent on behalf of your account, so send it only when your account can make that attestation. Only the account that owns the voice may set it, with its API key or a temporary token minted with that key; another account is refused with `not_owner`. Your account must also have accepted the [biometric data terms](#biometric-data-terms).
```python
# Continues the program above. You supply play().
renew = aai.TTSConfig(voice_id=os.environ["ASSEMBLYAI_VOICE_ID"], extend_biometric_consent=True)
with aai.TTSSession(renew) as session: # a session
for chunk in session.speak("Consent renewed."):
play(chunk)
print(session.termination.voice_expires_at) # the expiry after the renewal
response = aai.TextToSpeech().synthesize("Consent renewed.", renew) # or an HTTP request
print(response.voice_expires_at)
```
```javascript
// Continues the program above. You supply play().
const session = client.tts.session({ voiceId: process.env.ASSEMBLYAI_VOICE_ID, extendBiometricConsent: true }); // a session
await session.connect();
for await (const chunk of session.speak("Consent renewed.")) play(chunk);
const summary = await session.close();
console.log(summary?.voice_expires_at); // the expiry after the renewal
const response = await client.tts.synthesize({ // or an HTTP request
text: "Consent renewed.",
voice_id: process.env.ASSEMBLYAI_VOICE_ID,
extend_biometric_consent: true,
});
console.log(response.voiceExpiresAt);
```
```bash
curl -sS --fail-with-body -D headers.txt https://tts.us.assemblyai.com/v1/speech \
-H "Authorization: $ASSEMBLYAI_API_KEY" \
-d '{"text": "Consent renewed.", "voice_id": "'"$ASSEMBLYAI_VOICE_ID"'", "extend_biometric_consent": true, "stream": false}' \
-o renewed.pcm
grep -i x-tts-voice-expires-at headers.txt
```
On the WebSocket URL, add `extend_biometric_consent=true` to the query string. On the OpenAI-compatible route, add `"extend_biometric_consent": true` to the request body, with the voice in `voice` as `{"id": ...}` or as its ID; see [OpenAI compatibility](/tts/openai-compatibility#extensions).
The renewal runs once the session or request has been accepted, and the new expiry shows in `Termination.voice_expires_at` and in `X-TTS-Voice-Expires-At` on a buffered HTTP response; a streamed response's headers leave with the first audio and still show the old expiry. A renewal within a day of the last one leaves the expiry as it is, and `Begin.configuration.extend_biometric_consent` echoes the flag without promising that the expiry moved. Compare the expiry before and after when you need to know.
## Delete a voice
Delete a cloned voice in the [voice catalog](https://www.assemblyai.com/dashboard/text-to-speech/voices) of the AssemblyAI dashboard, where every voice clone of your account is managed. Alternatively, call the voices API directly to delete a voice you own: `DELETE /v1/voices/{voice}` on the host of the voice's [data zone](#data-zones), with your API key in `Authorization`, where `{voice}` is the voice's slug.
```bash
curl -sS --fail-with-body -X DELETE "https://tts.us.assemblyai.com/v1/voices/$ASSEMBLYAI_VOICE_ID" \
-H "Authorization: $ASSEMBLYAI_API_KEY" \
-o /dev/null -w '%{http_code}\n'
```
```python
import os
import httpx # ships with the Python SDK
response = httpx.delete(
f"https://tts.us.assemblyai.com/v1/voices/{os.environ['ASSEMBLYAI_VOICE_ID']}",
headers={"Authorization": os.environ["ASSEMBLYAI_API_KEY"]},
)
response.raise_for_status() # 204, no body
print("deleted")
```
```javascript
const response = await fetch(`https://tts.us.assemblyai.com/v1/voices/${process.env.ASSEMBLYAI_VOICE_ID}`, {
method: "DELETE",
headers: { Authorization: process.env.ASSEMBLYAI_API_KEY },
});
if (response.status !== 204) throw new Error(`${response.status}: ${await response.text()}`);
console.log("deleted");
```
Success is `204` with no body. A voice you can use but do not own, such as an AssemblyAI voice or another account's public voice, answers `403` (`voice_readonly`), a voice your account cannot see `404` (`voice_not_found`), and a [temporary token](/tts/authenticate-with-a-temporary-token) `403` (`temp_token_not_allowed`). A repeated delete by the same slug answers `404`, as the deleted voice no longer holds the slug; once you give the slug to a new voice, it names that voice.
Deletion cannot be undone. What it does:
- **New sessions and requests with the voice are refused** within 6 minutes, in most cases at once, with `voice_not_found`, the same answer as for a voice that never existed. See [Errors](#errors).
- **Sessions already open keep speaking.** The voice is checked when a session opens, so a session that has the voice finishes its conversation with it. `Terminate` the sessions yourself if they should stop.
- **The slug is free again** for a new voice of yours.
- **Its data is removed.** The sample recording and the voice model are deleted from storage afterwards, normally within two days.
A voice belongs to one data zone, so delete it on that zone's host: `tts.us.assemblyai.com` for a voice created in the US, `tts.eu.assemblyai.com` for one created in the EU. On the other host it is not found.
## Biometric data terms
A session or request with one of your account's own cloned voices, or with `extend_biometric_consent`, needs your account to have accepted AssemblyAI's biometric data terms. Accept them on the [data controls page](https://www.assemblyai.com/dashboard/settings/data-controls) of the AssemblyAI dashboard. AssemblyAI voices and other accounts' public voices never need them.
The server checks the terms after the voice checks: before `Begin` on the WebSocket API, and before the first audio on the HTTP API.
| Outcome | WebSocket API | HTTP API | OpenAI-compatible route |
|---------|---------------|----------|-------------------------|
| Your account has not accepted the terms | `3006` `biometric_terms_not_accepted` | `403` `biometric_terms_not_accepted` | `403` `permission_error`, code `biometric_terms_not_accepted` |
| The terms could not be checked just now | `1011` `auth_unavailable` | `503` `auth_unavailable`, `Retry-After: 1` | `503` `service_unavailable_error`, code `auth_unavailable` |
The refusal has `retryable` `false`, and `param` is `voice_id`, or `extend_biometric_consent` when you set it; on the OpenAI-compatible route, a refusal for the voice names the field that named it, `voice` or `voice_id`. Its text names the page where the terms are accepted:
```text
This account has not accepted the biometric data terms. Please visit https://www.assemblyai.com/dashboard/settings/data-controls to review and accept them.
```
On the WebSocket API, read the text in the `Error` frame: the close reason is `See Error message for details`. Accept the terms, then open the session or send the request again. `auth_unavailable` is transient: retry after a short backoff. With a [temporary token](/tts/authenticate-with-a-temporary-token), mint a new token for each retry, as for every connection, reconnects included.
## Errors
A voice the session cannot use is refused before `Begin`, with an `Error` frame in its place; over HTTP the same refusals are `422` answers with the same codes, except where noted, and the SDKs raise them as the tabs above describe. Branch on the code, not the text:
- **`voice_not_found`** (close `3012`): no voice with this ID or slug that your account can use on this host. It does not exist, was [deleted](#delete-a-voice), is another account's private voice, or belongs to the other [data zone](#data-zones). Check the ID and the host.
- **`voice_expired`** (close `3012`): the [consent period](#consent-period) ended. Create the voice again.
- **`voice_blocked`** (close `3012`): the voice is blocked. Contact support.
- **`voice_not_paired`** (close `3014`): the voice is not ready on the session's model, usually because it was just created. Try again shortly; if it keeps failing, check the voice in the dashboard. The texts of this refusal are in [Error handling](/tts/error-handling#the-texts-of-3014).
- **`not_owner`** (close `3006`): `extend_biometric_consent` was set on a voice another account owns. Leave it out.
- **`biometric_terms_not_accepted`** (close `3006`, `403` over HTTP): your account has not accepted the [biometric data terms](#biometric-data-terms). Accept them on the dashboard's data controls page.
- **`invalid_parameter` on `voice_id`** (close `3006`): the value is neither a voice ID nor a slug, or it is a slug with capital letters. Send the value the dashboard shows, a slug in lowercase.
- **`model_not_found`**, **`model_retired`** (close `3006`, `3014`): a `model_id` the region does not know, or a retired model. Leave `model_id` out.
None of these is retryable unchanged, except `voice_not_paired` once the voice is ready. When the biometric data terms cannot be checked just now, the session closes `1011` `auth_unavailable` instead (`503 auth_unavailable` over HTTP), which you can retry. A session refused for its voice still counts toward your account's [new sessions per minute](/tts/limits). [Error handling](/tts/error-handling) has every close code and what to do about it.
## Related
- [Voices](/tts/voices): the voice catalog and the AssemblyAI voices to start with.
- [WebSocket API basics](/tts/connect-parameters): how `voice_id` and `extend_biometric_consent` are checked at connect.
- [Error handling](/tts/error-handling): every close code and what to do about it.
- [OpenAI compatibility](/tts/openai-compatibility#cloned-voices): a cloned voice through the OpenAI SDKs.
- [Cloud endpoints and data residency](/tts/endpoints-and-data-zones): the US and EU hosts.
---
# Prompting guide
URL: https://www.assemblyai.com/docs/tts/prompting-guide
Source: docs/tts/prompting-guide.mdx
Navigation: Text to Speech > Features
Description: How the text you send shapes the speech, from pacing and tone to how numbers, dates and other entities are read, and how to change it.
AssemblyAI TTS natively handles pacing, tone and normalization from the input text. For most sentences, the input text should match what you expect to hear, without special customization. Use this guide to understand and adapt the built-in handling of pace, tone and entity pronunciation.
## Pacing and tone
Use full sentences with proper punctuation. When the input text has no punctuation, the model may not render the speech into the sentences you expect to hear. To control pace and tone:
- **Emphasis:** use exclamation marks to emphasize a statement.
- **Questions:** use question marks to apply the correct tone for a question.
- **Surprise:** follow a question mark with an exclamation mark to add a tone of surprise to the question. Compare `You haven't seen this before?` with `You haven't seen this before?!`.
- **Short pauses:** use commas or ellipses to produce a brief pause. Both produce a similar pause, and writing several together doesn't make the pause longer.
- **Sudden cutoffs:** end the text with a dash, written as two hyphens, to cut the speech off. Compare `Yes, it's right here in front of --` with `Yes, it's right here in front of.`.
- **Stressed words:** capitalize an entire word to emphasize it. Compare `You call THAT a spoon?` with `You call that a spoon?`.
- **Filler words:** write them directly into the text, separated by commas and without repeated consonants: `It's, uh, right here in front of the, uh, um, thing.`
- **Stutters:** use dashes: `I c-c-can't believe what I'm seeing!`
- **False starts:** repeat the first word of a sentence, followed by a dash and a space: `That- that is the one.`
## Writing replies that sound natural
Punctuation shapes how a sentence is read, but how the sentence is written matters more. A reply written the way a person talks on a call is read like one. When an LLM writes your agent's replies, give it these rules:
- **Open softly.** The most natural replies start mid-thought, with `Hi, um,`, `Yeah, so...` or `Oh, uh,`, rather than a clean, complete opening sentence.
- **Keep a light stumble.** A reply with every hesitation removed sounds too smooth. One small stumble per reply, such as a filler, a false start or a trailing `...`, does more for a natural delivery than fine-tuned punctuation.
- **Use fragments.** Short beats such as `I, uh,` or `So...` before the point are read well. Not every reply needs to be a full sentence.
- **Say the words, not the shorthand.** Write what a person would say aloud. An abbreviation that's spoken letter by letter only inside your company sounds wrong on a call, so write the full phrase, and keep only abbreviations that everyone says that way.
- **Keep the first reply short.** A long first turn, such as a checklist read out on hello, sounds scripted however it's punctuated. Ask one short question, and let the other person ask for detail.
### Before and after
Each pair says the same thing. The after version is written to be spoken.
**Confirming an appointment**
- Before: `Great, you're all set for Tuesday, 11/3/2026 at 4:30 PM with Dr. Patel. If anything changes, just call us at (555) 867-5309, okay?`
- After: `Perfect! You are all set for Tuesday, November 3rd at 4:30 PM with Dr. Patel. And if anything changes, just call us at (555) 867-5309. Okay?`
The date is written the way it's said, without the year. `And` carries the reply into its second sentence, and `Okay?` stands on its own as a check-in.
**Confirming a payment**
- Before: `Your current balance is $1,248.50, and it's due on the 15th. I'll charge the card ending in 4-3-2-1. Is that correct?`
- After: `So your current balance is $1,248.50, and it's due on the 15th, okay? I'll charge the card ending in 4-3-2-1. Is that the right one?`
`So` opens the reply mid-thought, `okay?` checks in before the next point, and `Is that the right one?` asks the way a person would, rather than `Is that correct?`.
**Looking something up**
- Before: `Hmm, let me look into that for you... Okay! I've, uh, sent a reset link to support@acme.com, so it should arrive in a minute or two.`
- After: `Okay, uh, one moment, uhh... let me take a look... I-I just sent you a password reset link to S-U-P-P-O-R-T@acme.com and it should arrive in a minute or two, okay?`
The wait is filled with fillers and trailing ellipses, `I-I` adds a stumble, and the address is spelled out letter by letter, the way you would read it to a caller.
## Normalization
Write currencies, phone numbers, addresses and other entities the way they're normally written. Before the model reads your text, the server writes the common ones out in words, in every language, and the model reads the rest the way they're normally pronounced:
| Entity | Input | Spoken as |
|---|---|---|
| Currency | `$12.13` | twelve dollars and thirteen cents |
| Phone number | `(415) 555-2876` | four one five, five five five, two eight seven six |
| Address | `54 Davis St` | fifty four Davis Street |
| Email address | `support@company.com` | support at company dot com |
| Web address | `example.com/help` | example dot com slash help |
| Date | `10/24/2026` | October twenty-fourth, twenty twenty-six |
| Measurement | `5 kg` | five kilograms |
| Shorthand (English) | `3rd Fl` | third floor |
| Symbol (English) | `±5` | plus or minus five |
| Phone key (English) | `Press * to repeat` | press star to repeat |
| Time | `4:30 PM` | four thirty p m |
| Common abbreviation | `USA` | U S A |
The server writes out amounts with a currency symbol, phone numbers, card and other ID numbers, long numbers after a label such as `order number`, email and web addresses, numbers with several dots such as IP addresses and versions, measurements with a unit symbol, dates written in numbers with their year, weekday abbreviations such as `Mon`, and US street addresses. In English it also writes out:
- shorthand such as `10 min`, `2 hrs`, `6'2"`, `3rd Fl`, `w/`, `c/o`, `p. 12` and `etc.`;
- symbols such as `±` and `≥`, a `~` before a number, read as "about", and a `*` or `#` after a word such as `press` or `dial`, read as "star" and "pound";
- file paths such as `~/.bashrc`, MAC addresses and hexadecimal values such as `0xFF`, symbol by symbol.
Other numbers, times, symbols and abbreviations reach the model as you wrote them, and the model reads them on its own.
A currency symbol next to a number makes it an amount: `$12.13` or `12,50 €` is read as an amount, rather than digit by digit.
Well-known abbreviations are read the expected way. For an uncommon acronym or abbreviation, put dashes between the letters: `E-T-O is Reference Evapotranspiration.`
### Reading numbers a specific way
When a number has to be spoken a particular way, such as when confirming a card number:
- **Digit by digit:** separate the digits with dashes. `4-3-2-1` is read as four three two one.
- **As a whole number:** write it with commas. `4,321` is read as four thousand three hundred and twenty one.
- **Short pauses between groups,** as in a card number: put a dash between the groups of digits. `4321-5678-9012-3456` is read digit by digit, with a short pause after each group of four.
## How text is read
Before the model reads your text, the server prepares it, in every language:
- It leaves out what is written but not said: emoji, Markdown and HTML markup, guillemets, footnote marks such as `†`, and list bullets. A run of `!` or `?` becomes one.
- It turns square, curly and angle brackets into round ones. The words inside them are read aloud: `[pause]` is read as the word "pause" and `[laughs]` as "laughs", not as instructions. Remove such tags before you send the text.
- It writes out the entities listed under [Normalization](#normalization).
- It ends the last sentence of each request with a period when that sentence has no closing punctuation mark.
Everything else reaches the model as written. Where the reading matters, write it out in words: a written form the model has not seen often can be read another way than you meant.
- Long text is split into pieces before it is spoken, and each piece is spoken on its own, so a split can be heard as a short pause. The server splits:
- at sentence ends: after `.`, `!` or `?`, with any closing quote or bracket, followed by whitespace, but not after a title such as `Dr.`, initials such as `e.g.` or `p.m.`, a German ordinal such as `3.`, or a list number that starts its line (`5.`);
- at every line break, so each line of a bullet list, a heading or a paragraph without a final period ends a piece;
- inside a long sentence, after `,`, `;` or `:` followed by whitespace, so `12,500` or `3:45` is never split;
- inside a long clause, between words, at whitespace or U+200B ZERO WIDTH SPACE;
- inside a long word, such as a hash or a long number, at the punctuation inside it (`/ . ? = & - _ : # @ + ~ , ;`), where its runs of digits start and end, and, if it is still too long, between characters. A letter is not separated from its accents, nor an emoji from its modifiers, unless one character with its marks is too long for a piece.
- Long text is never refused for its length or for a lack of spaces: a 400-letter run, a hexadecimal hash or a 300-digit number is spoken in pieces. The limits on input size are the character limits: 2000 characters per `Generate` and 8000 sent but not yet synthesized on the WebSocket API (see [Limits](/tts/limits)), and 5,000 per request on the [HTTP API](/tts/api-reference/http-api/overview).
- In rare cases a piece does not fit the model with the session's voice. The session then closes with `3006` after `Begin`, with a text that begins `Text contains a -character segment too long to synthesize` and ends `Send shorter sentences.` The server splits every text into pieces that fit, so this points to a fault on our side: report it with the session's `id`. Sending the text again in shorter sentences avoids it.
- Text with no letter or digit is not spoken. The server checks each sentence, or group of sentences, that it synthesizes at once:
- only whitespace and symbols, such as emoji, `$`, `€`, `+` or `©`: no audio;
- only punctuation, such as `?!`, `...`, `%`, `#`, `&`, `@` or `-`: a short silence of at most about 0.64 seconds;
- anything else is spoken.
`WordBoundaries` still lists your text exactly as you sent it, markup included; see [Word boundaries](/tts/word-boundaries).
## Punctuation and the first audio
Sentence ends also decide when streamed text starts playing. When you send a reply in several `Generate` messages as your LLM writes it, the server starts speaking at the first sentence end it holds: a `.`, `!` or `?` followed by a space or by more text. A sentence end within about the first 50 characters of a reply does not count, so a short first sentence such as `Hi there.` is spoken together with the sentence after it, and text without a sentence end waits for more text. A short reply sent in one `Generate` waits for your `Flush`.
So keep the punctuation your LLM writes, forward its tokens with their spaces as they arrive, and send `Flush` at the end of each sentence as it completes, so that a short first sentence starts at once. Always send `Flush` as soon as the reply is complete: everything the server holds is spoken then. See [Send text as it is written, and flush each sentence](/tts/getting-started/optimizing-accuracy-and-latency#send-text-as-it-is-written-and-flush-each-sentence).
## Related
- [Voices](/tts/voices): choose a voice and the language it speaks.
- [Word boundaries](/tts/word-boundaries): the time each word is spoken.
- [Optimizing accuracy and latency](/tts/getting-started/optimizing-accuracy-and-latency): the other habits that shorten the wait for the first audio.