TTS stands for text to speech: technology that converts written text into spoken audio. Modern TTS systems use neural networks to predict how a sentence should sound, including pronunciation, rhythm, and intonation, and then generate a waveform you can play. The same technology reads articles aloud, voices GPS directions, and lets AI voice agents talk back to callers in real time.
Quick note if you landed here from social media: "TTS" also gets used as slang (on TikTok it usually means the app's built-in text-to-speech voice). And in medicine it can stand for other things entirely. This guide is about the technology.
Most explanations of TTS stop at "it reads text out loud." That was fine when the main job was a screen reader or an audiobook. But the fastest-growing use of TTS today is conversation. Voice agents answer support lines, book appointments, and qualify leads. And a voice that sounds fine reading a paragraph can sound robotic the moment it has to talk to someone.
So this guide covers both: how text to speech works under the hood, and why conversational TTS is a different problem from read-aloud TTS.
What is text to speech?
Text to speech, also called speech synthesis, is the process of generating human-sounding speech from text. You give the system a string like "Your appointment is on Tuesday at 3:30pm," and it returns audio of a voice saying it.
TTS is the mirror image of speech-to-text, which turns audio into text. Speech-to-text is also called automatic speech recognition (ASR). Put the two together with a language model in the middle and you have the basic loop behind every voice agent: listen, think, speak.
A few terms you'll see used interchangeably:
- TTS / text to speech: the general category.
- Speech synthesis: the academic term for generating speech by machine.
- Neural TTS: TTS built on deep learning models, which is nearly all modern TTS.
- Voice: a specific speaker identity a TTS model can produce (age, accent, timbre, style).
How does text to speech work?
Every TTS system has to answer two questions. What should be said, exactly? And how should it sound? Modern systems split that into a pipeline. Some newer models merge steps, but the jobs stay the same.
1. Text normalization
Raw text is full of things nobody says the way they're written. "Dr. Lee" becomes "Doctor Lee." "$4.50" becomes "four dollars and fifty cents." "3:30pm" becomes "three thirty p m." Normalization expands numbers, dates, currencies, abbreviations, and symbols into words. It's also where a lot of real-world errors start. Is "St." saint or street? Is "1/2" a date or a fraction?
2. Phonemes and pronunciation
Next, the system works out how each word is pronounced, usually as phonemes (the basic sound units of a language). This step is called grapheme-to-phoneme conversion, or G2P. It handles words that are spelled the same but said differently, like "read" (present) and "read" (past). Brand names, surnames, and technical terms are where it most often goes wrong. That's why many TTS systems let you supply custom pronunciations.
3. Prosody and acoustic modeling
Prosody is the music of speech: pitch, timing, stress, and pauses. It's the difference between "You're coming?" and "You're coming." An acoustic model predicts these features and turns the phoneme sequence into an intermediate representation of sound, often a mel spectrogram (a picture of how energy is spread across frequencies over time).
This is the step that decides whether a voice sounds human or flat. Getting the words right is table stakes. Getting the emphasis right is the hard part.
4. Vocoder: generating the waveform
A neural vocoder turns that intermediate representation into an actual audio waveform, the samples your speaker plays. Early neural vocoders produced great audio but were slow. Modern ones run many times faster than real time, which is what makes live, streaming TTS possible.
5. Streaming the audio out
For read-aloud use cases, the system can generate the whole file and hand it back. For conversation, it streams audio in small chunks as soon as the first words are ready. How quickly that first chunk arrives matters more than how fast the whole sentence finishes.
From robotic to natural: a short history of TTS
- Rule-based and formant synthesis (1970s–1990s): voices built from hand-tuned acoustic rules. Intelligible, unmistakably robotic.
- Concatenative synthesis (1990s–2010s): stitching together tiny recorded snippets of a real speaker. More natural, but audible seams and almost no control over emotion or emphasis.
- Statistical parametric TTS (2000s–2010s): models that predicted speech parameters. Smoother and more flexible, but muffled.
- Neural TTS (mid-2010s onward): deep learning models for the acoustic model and vocoder. This is when synthetic voices started to sound convincingly human.
- Large, LLM-style speech models (2020s): models trained on huge amounts of speech that can follow style instructions, imitate a voice from a short sample, and handle expressive, conversational delivery.
TTS is only one third of a voice agent. Start with accurate speech-to-text and LLMs through one API key.
Read-aloud TTS vs. conversational TTS
Here's where most TTS explainers stop short. The same technology serves two very different jobs, and the requirements diverge fast.
| Read-aloud TTS | Conversational TTS | |
|---|---|---|
| Typical use | Screen readers, audiobooks, voiceovers, article narration | Voice agents, IVR, phone assistants, in-app voice |
| Input | Complete, edited text, available up front | LLM output arriving token by token, often mid-sentence |
| Latency budget | Seconds are fine | Tight. Human turn-taking gaps are roughly 200 ms |
| What "good" sounds like | Clear, consistent, pleasant over long stretches | Natural back-and-forth: short replies, varied intonation, the right emphasis |
| Interruptions | Not a concern | Must stop instantly when the caller talks over it |
| Hard content | Long paragraphs, chapter structure | Phone numbers, emails, confirmation codes, names, spelled-out IDs |
A few of these deserve more detail.
Latency is a feature
Research across languages has found that people typically leave a gap of about 200 milliseconds between turns in conversation (Stivers et al., PNAS, 2009). A voice agent can't match that end to end today. But every extra few hundred milliseconds of TTS delay makes a pause feel awkward and gets callers talking over the agent. That's why conversational TTS is judged on time to first audio, not total generation time.
The input isn't finished when speech starts
In a voice agent, the language model is still writing its reply when the TTS starts speaking. The TTS has to decide how to say the start of a sentence before it knows how the sentence ends. Systems that wait for full sentences sound better but feel slower. Systems that speak too early can land the stress in the wrong place.
Short replies are harder than long ones
It sounds backwards, but a two-word reply ("Got it.") gives the model almost no context to choose the right tone. Conversational speech is full of these: confirmations, acknowledgements, quick questions. A voice that's lovely on a paragraph can sound clipped or oddly cheerful on "Okay."
Numbers and identifiers have to be exact
Reading back a confirmation code or a phone number is where TTS errors cost real money. The caller has to hear "four-one-five" as three digits, in the right grouping, at a pace they can write down. Normalization and pacing matter as much as voice quality here.
What makes a TTS voice sound natural?
"Natural" gets used loosely. In practice it breaks down into a few things you can listen for:
- Prosody: does the emphasis land on the word that carries the meaning?
- Pacing and pauses: are there breaths and micro-pauses where a person would put them, or one even stream of words?
- Variation: does the same phrase sound slightly different each time, the way people do, or identical every time?
- Conversational texture: can it handle a light "um," a laugh, or a softer tone when delivering bad news, when you want it to?
- Pronunciation accuracy: names, brands, and domain terms said the way your customers say them.
- Consistency: the voice keeps the same identity across a 10-minute call, without drifting in pitch or accent.
One useful test: write down a reply your agent will say often, like "Sure, I can help with that. Can I get the email on the account?" Then listen to it 10 times. Flat, identical, or rushed delivery shows up fast.
How TTS is measured
There's no single score for a voice. Teams usually combine a few measures:
- Time to first audio (TTFA), or time to first byte: how long from sending text to receiving the first playable audio. This is the key latency number for conversation.
- Real-time factor: how long it takes to generate audio relative to the audio's length. Below 1.0 means faster than real time.
- Mean opinion score (MOS): listeners rate naturalness, usually on a 1–5 scale.
- Blind preference tests: listeners hear two voices saying the same thing and pick one. This is often more telling than absolute scores.
- Intelligibility (WER on re-transcription): run the generated audio back through speech-to-text and measure the word error rate. Mispronounced or slurred words show up as errors.
The catch: most of these are measured on read speech. A voice can score well reading sentences and still feel off in a live call. If you're building a voice agent, test voices inside a real conversation, not just on sample sentences. Our 2026 insights report on what makes a good voice agent looks at quality across the whole stack, not just one component.
Common uses of text to speech
- Voice agents and customer service: AI voice agents that answer calls, book appointments, collect payments, and route support. This is where conversational quality matters most.
- IVR and phone systems: menus and prompts, increasingly replaced by agents that can understand open-ended answers.
- Accessibility: screen readers and read-aloud tools for people with visual impairments, dyslexia, or other reading difficulties.
- Content and media: voiceovers, audiobooks, podcasts, and localized video.
- In-app and device voice: navigation directions, smart speakers, in-car assistants.
Where TTS fits in a voice agent
A voice agent runs a loop on every turn:
- Speech-to-text transcribes what the caller said, in real time.
- Turn detection decides when the caller has finished speaking.
- An LLM decides what to say or do next, and may call tools (look up an order, book a slot).
- Text to speech speaks the reply, streaming audio as the LLM writes it.
Each step adds latency and each can fail. A misheard email address in step 1 means a wrong lookup in step 3, no matter how good the voice sounds in step 4. That's why voice agent teams evaluate the whole loop, not just the TTS voice. If you want the deeper version, we wrote about where voice agent stacks start showing their limits and about the ongoing engineering cost of running separate STT, LLM, and TTS vendors.
Two ways teams build this today:
- Assemble it yourself: pick a speech-to-text API, an LLM, and a TTS API, then orchestrate them (often with an open-source framework like LiveKit Agents or Pipecat). Maximum control, more moving parts. Our guide to the top text-to-speech APIs compares the TTS options.
- Use a single voice agent API: one connection that handles listening, thinking, and speaking. AssemblyAI's Voice Agent API works this way. It's built on Universal-3.6 Pro Realtime for speech recognition, at a flat $4.50/hr.
One WebSocket for speech-to-text, LLM, and voice, so you can hear a full conversation loop without wiring three vendors together.
What to look for when choosing TTS for conversation
- Time to first audio under real network conditions, not just the vendor's best case.
- Streaming input: can it start speaking from partial LLM output?
- Interruption handling: how fast can playback stop and flush when the caller barges in?
- Voice range: enough voices, accents, and languages for your callers.
- Control: speed, custom pronunciation, and the ability to mark up emphasis or pauses.
- Word-level timestamps: so you know exactly what the caller heard before they interrupted.
- Behavior on hard strings: phone numbers, emails, dates, and codes.
- Cost at your volume: per-character pricing adds up fast across millions of agent turns.
The best test is still the simplest one. Put the voice in a real call flow, talk to it, and interrupt it. You'll learn more in five minutes than from any spec sheet.
Start with the speech-to-text and LLM layers of your voice agent today, with one key for both.
Frequently asked questions
What does TTS stand for?
TTS stands for text to speech, technology that converts written text into spoken audio. On social apps like TikTok, "TTS" usually refers to the app's built-in text-to-speech voice. In other fields, such as medicine, the same letters can stand for unrelated terms.
What is the difference between TTS and speech synthesis?
They mean essentially the same thing. Speech synthesis is the broader, academic term for producing speech by machine. Text to speech is the most common kind of speech synthesis, where the input is written text.
How does neural text to speech work?
Neural TTS normalizes the text, works out pronunciation, and predicts prosody (pitch, timing, stress) with a neural acoustic model. A neural vocoder then turns that prediction into an audio waveform. Newer models combine some of these steps, but the jobs are the same.
What is conversational TTS?
Conversational TTS is text to speech built for live dialogue, such as voice agents. It has to start speaking from partial input, deliver short replies naturally, read numbers and IDs exactly, and stop instantly when interrupted, all with low time to first audio.
What is time to first audio in TTS?
Time to first audio (TTFA) is the delay between sending text to a TTS system and receiving the first playable chunk of audio. For voice agents it matters more than total generation time, because it decides how long the caller waits in silence.
Is text to speech the same as speech recognition?
No. Text to speech turns text into audio. Speech recognition, or speech-to-text, does the opposite: it turns audio into text. Voice agents use both, with an LLM in between deciding what to say.
