Now live New models for transcription and text-to-speech. AssemblyAI-STT-3.7 AssemblyAI-TTS-1
Insights & Use Cases

Async transcription accuracy on hard audio: noisy calls, overlapping speakers, accents and filler words

How to tune AssemblyAI-STT-3.7 for noisy calls, overlapping speakers, accents and filler words, with the disfluencies, multichannel and keyterms parameters.

Abstract green sphere illustration

Written by

Kelsey Foster

Published on

19 November 2026

Most speech-to-text demos use clean audio: one speaker, a good mic, a quiet room. Production audio doesn't look like that. It's a call recording where the headset clips, two people talk over each other, a customer reads a card number while a TV plays, and someone says "I, uh, I mean the second one." That messy 20% of your audio decides whether your product works, and it's the part demos skip.

Most of it is tunable. AssemblyAI-STT-3.7, the flagship model on the pre-recorded Speech-to-Text API, gives you a parameter for each failure mode, so you choose the behavior instead of accepting a black box. Here's how to handle noise, crosstalk, accents, filler words and the names and numbers your workflow depends on, with the published numbers behind each one.

Why clean-audio WER doesn't predict production accuracy

A headline word error rate on a benchmark tells you how a model handles tidy speech. It doesn't tell you whether it gets the account number, the speaker of the closing commitment, or the Spanish sentence in the middle of an English call. Those are the tokens a wrong transcript makes expensive, and they cluster in the hard audio.

So judge a model on the specific failures your audio produces:

Hard-audio failure What goes wrong AssemblyAI-STT-3.7 lever
Background noise Phantom words, clipped words, late timestamps Send the original signal; no client-side denoising
Overlapping speakers Lines credited to the wrong person multichannel or speaker_labels
Accents and language switching Wrong language or script mid-call Native code switching; language_codes to restrict the set
Filler words and false starts Verbatim when you need clean, or the reverse disfluencies
Names, numbers, jargon Misspelled product names and identifiers keyterms_prompt and prompt

Background noise and low-quality recordings

Noise is the most common complaint and the most misunderstood one. The instinct is to clean the audio before it reaches the model by running aggressive noise cancellation first. Resist it. Client-side noise reduction adds artifacts, and those artifacts often confuse the model more than the original noise did, because they strip out the speech cues the model relies on.

For pre-recorded audio, send the cleanest original signal you have and let the model do the separation. Noise also hurts timing, which matters for captions and click-to-play search. On noisy English recordings, AssemblyAI-STT-3.7 cut the median word-start error from 59 ms on Universal-3.5 Pro to 40 ms, so a caption built on a noisy recording lands closer to when the word was spoken.

If you capture audio live, the lever moves to the server. Voice Focus isolates the primary voice and suppresses background chatter, keyboard clicks and room echo before the audio reaches the model. It has a near-field setting for headsets and handsets and a far-field setting for conference rooms and laptop mics. It's available on AssemblyAI-STT-3.7-Realtime-Agent and Universal-3.6 Pro Realtime, and AssemblyAI-STT-3.7-Realtime-Agent transcribes 47% less background speech than Universal-3.6 Pro Realtime (1.34 vs 2.53 words per 1,000).

Overlapping speakers and crosstalk

Two people talking at once is hard, and a sensitive model makes one version of it harder: it picks up audio bleeding between channels. A common failure goes like this. The agent's headset echoes the customer's voice back into the agent's mic, the echo gets transcribed as extra lines on the agent's side, and any downstream QA that assumes one speaker per channel breaks.

The fix is architectural. If your call stack records each party on its own channel, use multichannel transcription: each speaker is transcribed from their own channel, so bleed can't be credited to the wrong person. When you only have one mixed track, use speaker diarization to label who spoke when. Reach for multichannel when you have the channels and diarization when you don't. If a single channel holds several people, such as a conference phone, combine the two and diarize within each channel.

AssemblyAI-STT-3.7 ships with a new diarization model, and it moves the numbers that decide whether a transcript reads right:

  • Diarization error rate (DER): 10.7% averaged over six English conversation datasets, the lowest of the pre-recorded APIs measured, ahead of ElevenLabs Scribe v3 (12.0%), Speechmatics Enhanced (16.2%), Gladia Solaria-1 (16.7%), AWS Transcribe (16.9%), Deepgram Nova-3 (17.4%) and Soniox (24.2%).
  • Words plus speaker together (cpWER): down 20% compared with Universal-3.5 Pro at launch.
  • Speaker count: speaker-count errors down 63%, so the transcript shows the right number of people far more often.
  • Short clips: on 60-second clips, cpWER fell from 34.0% to 18.6%, which matters for voice notes and call snippets where the model has little audio to learn each voice.

The new model weighs its own confidence when it assigns each word to a speaker, so sentences stay whole while a quick "yeah" or "mm-hmm" still goes to the person who said it. If you already set speaker_labels: true, you get it with no code change.

Transcribe Hard Audio Accurately

Run AssemblyAI-STT-3.7 on your noisiest, most overlapping call recordings and check the speaker labels yourself. New accounts get free credits to start.

Sign up free

Accents and code switching

Accented speech breaks transcripts in two opposite ways. Some systems force a whole call into one language, writing Catalan as Spanish. Others flip into another language or script when the speaker never switched, often because of an accent. You need both behaviors handled at once: follow every real switch, and stay put when the speaker does.

AssemblyAI-STT-3.7 handles code switching natively across its 32 languages, with nothing to turn on. On English benchmark audio, foreign-script characters in the transcript fell from 160.8 to 23.6 per million characters compared with Universal-3.5 Pro, so accented English stays in English. On audio that mixes English with German, Spanish, French, Italian or Portuguese, it has the lowest average error rate measured, 6.2%, ahead of ElevenLabs Scribe v3 (6.8%), Soniox (10.2%), Azure Batch Transcription (17.4%), AWS Transcribe (22.9%), OpenAI GPT-4o Transcribe (44.7%) and Deepgram Nova-3 (49.4%).

If you know which languages a call can contain, restrict the set with language_codes, for example ["en", "es"]. For languages outside the 32, add universal-2 to speech_models for coverage across 99 languages.

Filler words and disfluencies: verbatim vs. clean

"Um," "uh," "you know," false starts and self-corrections are where teams disagree, because there's no single right answer. A legal record or a research interview may need every hesitation. A meeting summary wants them gone. An audio editor needs the transcript to match the recording word for word, so cutting a word cuts the right sound.

On AssemblyAI-STT-3.7 you set this with one parameter, disfluencies, not with prompting. The docs are explicit that formatting and behavioral instructions in prompt are ignored, so a prompt asking the model to keep or drop fillers does nothing. Here's how the disfluencies parameter behaves:

  • It defaults to true whenever assemblyai-stt-3-7 appears anywhere in speech_models, even in ["universal-2", "assemblyai-stt-3-7"]. That's a change from Universal-3.5 Pro, which removed fillers by default.
  • true keeps more than fillers on 3.7: repetitions, stutters, false starts and informal contractions such as "gonna" and "kinda" stay in. On Universal-2, true keeps filler words only.
  • false gives you a clean read. Set it explicitly if you migrated from Universal-3.5 Pro and your downstream code expects clean text.

The research post measures this with F1 scores against verbatim references. With disfluencies on, disfluency F1 is 75.0 and repetition F1 is 72.1; with it off, they drop to 34.1 and 41.4, which is the setting doing its job.

A second parameter, format_text, controls written vs. spoken form. It defaults to true ("$12.50", "jane.doe@example.com"); set it to false for spoken form ("twelve fifty", "jane dot doe at example dot com"). The two are independent, so a verbatim, spoken-form transcript for a legal record is two booleans.

Getting names, numbers and jargon right

The highest-return lever on hard audio is telling the model which words to expect. Keyterms prompting takes up to 1,000 words or phrases on AssemblyAI-STT-3.7, each up to six words: product names, agent names, plan names, identifier formats. A keyterm turns a guess into the right spelling, even on poor audio.

The contextual prompt parameter does a different job. It describes what the audio is about, such as "Customer support call about a mobile phone plan upgrade", in plain sentences. Describe the audio rather than giving the model instructions, and keep lists of terms in keyterms_prompt instead.

A starting configuration for call-center audio

Putting it together for two-party call analytics audio with agent and customer on separate channels:

from assemblyai.prerecorded.v2 import Transcriber, TranscriptionConfig

config = TranscriptionConfig(
    speech_models=["assemblyai-stt-3-7", "universal-2"],  # 3.7 first, Universal-2 for other languages
    language_detection=True,
    multichannel=True,      # agent and customer on separate channels
    disfluencies=False,     # clean read; omit it to keep fillers (the 3.7 default)
    keyterms_prompt=["Northwind Mobile", "Family Share 40", "IMEI"],
)

transcriber = Transcriber(api_key="<YOUR_API_KEY>", config=config)
transcript = transcriber.transcribe("./support-call.wav")

if transcript.status == "error":
    raise RuntimeError(f"Transcription failed: {transcript.error}")

for utterance in transcript.utterances:
    print(f"Channel {utterance.channel}: {utterance.text}")

If you only have a single mixed track, swap multichannel=True for diarization and give the model a speaker range instead of an exact count:

config = TranscriptionConfig(
    speech_models=["assemblyai-stt-3-7"],
    speaker_labels=True,
    speaker_options={"min_speakers_expected": 2, "max_speakers_expected": 4},
)

Set max_speakers_expected a little above the number you expect; setting it too high can split one speaker into several labels. AssemblyAI-STT-3.7 costs $0.21/hr of audio, the same as Universal-3.5 Pro, and you can check volume rates on the pricing page.

Try It on Your Hardest Recording

Upload a tough file in the playground and compare the transcript yourself: noise, crosstalk, accents and all.

Try playground

Measure it on your own truth files

Don't treat any vendor's benchmark, including ours, as a stand-in for your data. Take a representative sample of your hardest recordings, create reference transcripts, and score the model against them. Our guides on how to evaluate speech recognition models and why raw WER can mislead walk through how to do it without fooling yourself.

Two details catch teams out. First, match your reference style to your settings: if the references keep fillers, score with disfluencies on, and if they spell numbers out, score with format_text off. Second, score speaker attribution separately from words with cpWER or DER, because a transcript can have every word right and the wrong person saying it.

Tune for your worst 20%

Hard audio feels like a wall, but most of it is a set of parameters you haven't set yet. Noise wants the original signal, plus Voice Focus if you capture live. Overlap wants multichannel or speaker_labels, depending on your source. Accents and mixed languages are handled natively, with language_codes if you know the set. Filler words want disfluencies, not a prompt. Names and numbers want keyterms_prompt. Tune those against your own hardest files and the messy audio stops breaking your product. For why the words users notice matter more than aggregate scores, see accuracy versus quality.

Transcribe Call Center Audio Accurately

Built for real-world contact center audio. Start with clear docs and a free account, and bring your hardest recordings.

Sign up free

Frequently Asked Questions (FAQs)

How does AssemblyAI-STT-3.7 handle accents and noisy audio?

AssemblyAI-STT-3.7 handles accented and mixed-language speech natively across 32 languages, and on English audio it cut foreign-script characters from 160.8 to 23.6 per million compared with Universal-3.5 Pro. For noisy recordings, send the original signal rather than pre-cleaning it, since client-side denoising adds artifacts. Its word timestamps are also tighter on noisy English audio, with median word-start error down from 59 ms to 40 ms.

How do I improve speech-to-text accuracy on hard audio?

Match each failure to a parameter: multichannel or speaker_labels for overlapping speakers, keyterms_prompt for names and identifiers, disfluencies for verbatim or clean output, and language_codes when you know the languages. Then measure on a sample of your own hardest recordings with reference transcripts. That combination moves accuracy more than switching settings blind.

Should I remove background noise before sending audio to a transcription API?

Usually not. Aggressive client-side noise cancellation introduces artifacts that often confuse the model more than the original noise. For live audio, use server-side Voice Focus on AssemblyAI-STT-3.7-Realtime-Agent instead of pre-cleaning.

What's the difference between multichannel transcription and speaker diarization?

Multichannel transcription transcribes each speaker from their own audio channel, which stops cross-channel bleed from being credited to the wrong person. Speaker diarization labels who spoke when inside a single mixed channel. Use multichannel when your recordings already separate agent and customer, diarization when you have one combined track, and both together when one channel holds several people.

How do you get a verbatim transcript with filler words?

On AssemblyAI-STT-3.7, verbatim is the default: disfluencies is true whenever assemblyai-stt-3-7 is in speech_models, and it keeps fillers, repetitions, stutters, false starts and informal contractions. Set disfluencies to false for a clean read. Prompting doesn't control this, because the model ignores formatting and behavioral instructions in prompt.

How much does AssemblyAI-STT-3.7 cost for pre-recorded audio?

AssemblyAI-STT-3.7 costs $0.21 per hour of audio on the pre-recorded API, the same price as Universal-3.5 Pro. New accounts get $50 in free credits for the speech APIs, and the pricing page lists volume rates and any add-on pricing.