How to write text that makes TTS sound human: fillers, false starts and pacing

Most voice agents that sound robotic aren't let down by the voice. They're let down by the text. An LLM writes a reply the way it writes an email: complete sentences, no hesitation, every detail packed into the first turn. Read aloud, that sounds exactly like what it is, a script.

With AssemblyAI's Text-to-Speech API, the input text is the prompt. There's no markup language to learn and no style tags to tune. The model takes its pacing, tone and hesitations from how the text is written, so the most effective thing you can do for naturalness happens before the API call, in the words your agent sends.

This guide covers the punctuation the model responds to, how to write fillers, stutters and false starts, how numbers and addresses get read, and a system prompt you can hand your LLM so every reply comes out written to be spoken. Everything here applies to AssemblyAI-TTS-1, and the full reference is in the prompting guide in the docs.

Why sentence structure matters more than markup

AssemblyAI-TTS-1 is trained on millions of hours of audio, including real conversational data. That shapes how you should write for it. You don't steer it with stage directions; you write the reply the way a person would say it on a call, and it reads it that way.

In practice, sentence structure carries more of the delivery than punctuation does. A reply that opens mid-thought, checks in with the caller, and carries one small stumble sounds natural even with ordinary punctuation. A perfectly punctuated reply that reads like a policy document still sounds scripted.

So there are two levels to work at:

1. The words and structure of the reply. This is the bigger lever, and it's mostly a job for your LLM's system prompt.
2. Punctuation and spelling choices. These fine-tune emphasis, pauses and how specific entities are read.

One thing that doesn't work: bracketed tags. [laughs], [pause] and similar stage directions are read aloud as words. The server turns square brackets into round ones and speaks what's inside, so strip any tags your LLM adds before you send the text.

Punctuation that changes delivery

Use full sentences with proper punctuation. Text with no punctuation may not be read in the sentences you expect. Beyond that, a few marks have specific effects:

A few practical notes:

• Stacking doesn't scale. !!! collapses to one !, and ,,, or ...... gives the same short pause as one comma or ellipsis. If you want a longer gap, end the sentence and start a new one.
• The surprise effect is subtle. ?! adds surprise per the docs, but in side-by-side listening the difference can be small. Don't build a feature on it.
• Use CAPS sparingly. One stressed word in a reply reads as intent. Three reads as shouting.

One observation from internal testing, not documented behavior: with some voices, replies carried more upward inflection than the tester wanted. Limiting question marks to real questions, and using a double period at the end of a statement, toned it down for them. Treat that as something to try on your chosen voice, and listen before you adopt it.

Fillers, false starts and stutters

This is where a model built for conversation earns its keep. AssemblyAI-TTS-1 handles disfluencies well when you write them the way they're documented:

• Fillers go in as plain words between commas, with no repeated consonants: It's, uh, right here in front of the, uh, um, thing. Write um, not ummm.
• Stutters use dashes inside the word: I c-c-can't believe what I'm seeing!
• False starts repeat the first word, then a dash and a space: That- that is the one.
• Trailing off uses an ellipsis: So... I think that's everything.

You can combine them. A filler inside a false start, like Is- uh, is that the right one?, often lands as a natural self-correction. Short words that people use to carry a conversation, such as Okay, And, So and Yeah, are reliable too.

Voices differ in how they render these. Some voices hold a long uhh... for a natural beat; others sound clipped on the same text. Once you've picked a voice, listen to your agent's most common replies with and without disfluencies before you commit to a style. Our guide to evaluating TTS for voice agents covers how to build that listening test.

Writing replies that sound natural

These five rules come straight from the docs, and they're the ones that move the needle most:

1. Open softly. The most natural replies start mid-thought: Hi, um,, Yeah, so... or Oh, uh,, not a clean, complete opening sentence.
2. Keep a light stumble. A reply with every hesitation removed sounds too smooth. One small stumble per reply, such as a filler, a false start or a trailing ..., does more for a natural delivery than fine-tuned punctuation.
3. Use fragments. Short beats such as I, uh, or So... before the point read well. Not every reply needs to be a full sentence.
4. Say the words, not the shorthand. An abbreviation your team says letter by letter can come out as a sound on a call. Write the full phrase, and keep only abbreviations everyone says the same way.
5. Keep the first reply short. A long first turn, like a checklist read out on hello, sounds scripted however it's punctuated. Ask one short question, and let the caller ask for detail.

Before and after

Each pair says the same thing. The after version is written to be spoken.

Confirming an appointment

• Before: Great, you're all set for Tuesday, 11/3/2026 at 4:30 PM with Jordan for a haircut. If anything changes, just call us at (555) 867-5309, okay?
• After: Perfect! You are all set for Tuesday, November 3rd at 4:30 PM with Jordan. And if anything changes, just call us at (555) 867-5309. Okay?

The date is written the way it's said, without the year. And carries the reply into its second sentence, and Okay? stands on its own as a check-in.

Confirming a payment (from the docs)

• Before: Your current balance is $1,248.50, and it's due on the 15th. I'll charge the card ending in 4-3-2-1. Is that correct?
• After: So your current balance is $1,248.50, and it's due on the 15th, okay? I'll charge the card ending in 4-3-2-1. Is that the right one?

So opens mid-thought, okay? checks in before the next point, and Is that the right one? asks the way a person would.

Checking an order

• Before: I have located your order. It shipped on 10/24/2026 via ground shipping and the estimated delivery date is 10/29/2026.
• After: Okay, so, I found it. It shipped on October 24th, and it should- it should be there by Thursday.

The lookup becomes a beat (Okay, so,), the dates lose their years, and one false start keeps it from sounding read off a screen.

Changing a plan

• Before: Your plan has been updated to the Pro tier, effective immediately. The new monthly charge of $49.00 will appear on your next statement.
• After: Done! You're on Pro now. Uh, your next bill will be $49, and that starts with your next statement.

Short, upbeat confirmation first, then the detail, with one filler before the number.

Numbers, dates, emails and abbreviations

Before the model reads your text, the server writes common entities out in words, in every language. Write them the way they're normally written:

When a number has to be read a particular way, make it explicit:

• Digit by digit: separate the digits with dashes. 4-3-2-1 is read as four three two one, which is what you want for card endings, PINs and confirmation codes.
• As a whole number: write it with commas. 4,321 is read as four thousand three hundred and twenty one.
• Grouped: put a dash between groups. 4321-5678-9012-3456 is read digit by digit, with a short pause after each group of four.

For uncommon acronyms, put dashes between the letters (E-T-O). For an email address you want spelled out to a caller, spell the part that matters: S-U-P-P-O-R-T@acme.com. And for dates, decide whether the year belongs in the sentence; a person confirming an appointment next week wouldn't say it.

Emoji, Markdown and HTML are dropped before synthesis, so a stray **bold** from your LLM won't be read out. It's still worth stripping them, because your word timings list the text exactly as you sent it.

When not to add disfluencies

Fillers and false starts are seasoning. Leave them out when:

• The detail is the point. Keep the card ending, the confirmation code or the amount clean. Put the stumble before the number, never inside it.
• The content is legal or compliance language. Disclosures and consent prompts should be read plainly and identically every time.
• The reply is already long. One stumble per reply is the guideline. Stacking three fillers into a long answer makes the agent sound unsure.
• The caller is frustrated or in a hurry. Short, direct replies read as competence. Keep the soft open, drop the hesitation.
• The voice doesn't carry them well. If your chosen voice clips fillers, use structure (So, And, Okay?) instead.

Give your LLM these rules

The rules above only work if they're applied to every reply, and your LLM writes every reply. Put them in the system prompt. Here's a starting point you can adapt:

You are speaking on a phone call. Everything you write is read aloud by a
text-to-speech model, so write the way a person talks, not the way they type.

- Start most replies mid-thought: "Yeah, so...", "Okay,", "Oh, uh,".
- Include at most one small stumble per reply: a filler ("uh", "um"), a false
  start ("That- that's the one."), or a trailing "...". Vary which one you use.
  Never put a stumble inside a number, code, or name.
- Write fillers as plain words between commas. Write "um", never "ummm".
- Use fragments when they sound natural. Not every reply needs a full sentence.
- Keep the first reply of the call to one short sentence or question.
- Use question marks only for real questions.
- Write dates the way they are said ("Tuesday, November 3rd"), without the year
  unless it matters.
- Read card numbers and codes digit by digit with dashes: 4-3-2-1.
- Write abbreviations as full words unless everyone says them letter by letter.
- Never use brackets, stage directions, emoji, Markdown, or lists.

Two notes on that prompt. First, the "vary which one you use" line matters: left alone, LLMs latch onto one filler and repeat it every turn, which is its own kind of robotic. Second, ask for plain prose explicitly. Models trained to produce chat answers default to bullet lists and bold text, and that's the fastest way to get a reply that sounds read.

If you're using Claude to drive your agent, our guide to prompting Claude to build voice agents goes deeper on system prompt structure for voice.

Send the text the way it's written

How you stream the text matters as much as what it says. The server starts speaking at the first sentence end it holds, so forward your LLM's tokens as they arrive, keep the spaces between them, and end each sentence with a flush. A speak() call per sentence does the same thing:

import os

import assemblyai as aai


def clean(text):
    # Stage directions such as [laughs] are read aloud, so drop them.
    out, depth = [], 0
    for ch in text:
        if ch == "[":
            depth += 1
        elif ch == "]" and depth:
            depth -= 1
        elif not depth:
            out.append(ch)
    return "".join(out)


def sentences(tokens):
    # Join LLM tokens and yield each sentence as soon as it ends.
    buffer = ""
    for token in tokens:
        for ch in token:
            buffer += ch
            if ch.isspace() and buffer.rstrip()[-1:] in (".", "!", "?"):
                if buffer.strip():
                    yield buffer.strip()
                buffer = ""
    if buffer.strip():
        yield buffer.strip()


def speak_reply(session, tokens, play):
    for sentence in sentences(tokens):
        for chunk in session.speak(clean(sentence)):  # raw 24 kHz 16-bit mono PCM
            play(chunk)


config = aai.TTSConfig(voice_id="49c443a4-8a1d-5f7e-b70b-98bdbd9823c3", language="english")  # Audrey

with aai.TTSSession(config, api_key=os.environ["ASSEMBLYAI_API_KEY"]) as session:
    reply = ["Yeah, ", "so... ", "I found it. ", "It should- ", "it should be there by Thursday."]
    with open("reply.pcm", "wb") as out:
        speak_reply(session, reply, out.write)

In production, reply is the token stream from your LLM and play writes to your audio output. Keep the first sentence short, so the first audio starts sooner. For the full streaming walkthrough, including generate(), flush() and cancelling on barge-in, see how to stream text to speech in Python over a WebSocket, and for how sentence boundaries affect the wait before the first audio, see TTS latency explained.

The text is the interface

It's tempting to treat naturalness as a property of the voice model alone. It isn't. The model does a lot, but the biggest gains here come from changing what your agent says: a softer open, one honest stumble, a date said the way people say dates. Those changes cost nothing at inference time, and they compound across every turn of every call.

If you're tuning the listening side too, the same idea applies in reverse: prompts shape how speech-to-text hears your callers. Our practical guide to prompt engineering for Universal-3 Pro covers that half of the conversation, and what is text-to-speech covers the basics if you're new to TTS.

Frequently Asked Questions (FAQs)

How do you make text-to-speech sound more natural?

Write the text the way a person talks on a call. Open replies mid-thought ("Yeah, so..."), keep one small stumble such as a filler or false start, use short fragments, and keep the first reply brief. With a conversational model like AssemblyAI-TTS-1, sentence structure does more for naturalness than punctuation or markup.

Can text-to-speech say filler words like um and uh?

Yes. With AssemblyAI TTS, write fillers as plain words between commas, such as "It's, uh, right here," with no repeated consonants ("um", not "ummm"). How long a filler is held varies by voice, so listen to your chosen voice before settling on a style.

How do I add a pause in TTS?

Use a comma or an ellipsis for a short pause. Several in a row don't make the pause longer, so for a longer gap, end the sentence and start a new one. Bracketed tags like [pause] don't work: AssemblyAI TTS reads the word inside the brackets aloud, so remove them before sending.

How do I make TTS read a number digit by digit?

Put dashes between the digits. AssemblyAI TTS reads "4-3-2-1" as four three two one, and "4,321" as four thousand three hundred and twenty one. For card numbers, put a dash between each group of digits to add a short pause after each group.

Do I need SSML to control AssemblyAI TTS?

No. AssemblyAI TTS takes pacing and tone from the text itself: exclamation marks for emphasis, question marks for question tone, a trailing double hyphen for a cutoff, and an all-caps word for stress. Markup, emoji and bracketed tags aren't used as instructions.

How should I prompt an LLM to write replies for a voice agent?

Give it explicit speaking rules in the system prompt: start mid-thought, allow at most one varied stumble per reply, keep the first reply short, write dates and numbers the way they're said, and never use lists, Markdown or brackets. Then stream its tokens to the TTS session as they arrive and end each sentence with a flush.