Speech-to-text prompting with AssemblyAI Universal-3.5 Pro

Most of the market believes speech-to-text prompting doesn't work. The conventional wisdom is that you either fine-tune a model or you live with whatever it hands back. Our benchmarks say otherwise — and this guide shows you exactly how to prompt Universal-3.5 Pro, the first promptable Speech Language Model, on pre-recorded audio.

This is the canonical guide for async (pre-recorded) prompting. If you're transcribing files — meetings, calls, podcasts, drive-through orders after the fact — you're in the right place. If you're building a live voice agent and want to prompt a stream in real time, read the companion guide on prompt engineering for real-time speech-to-text instead. More on that split below.

Two prompting tools, and they work together

Universal-3.5 Pro gives you two separate, complementary controls on pre-recorded audio. They are not mutually exclusive — the strongest results come from using both at once.

• prompt — a short natural-language description of what the audio is about. Think domain, scenario, or subject matter in 2–50 words. It sets context; it does not control formatting. Behavioral or formatting instructions are ignored.
• keyterms_prompt — an explicit list of up to 1,000 words or phrases (max six words per phrase) you want spelled and recognized exactly. This is your find-and-replace for names, SKUs, drugs, and jargon.

Use prompt to tell the model the world the audio lives in, and keyterms_prompt to lock down the exact spellings inside that world. Here's both together in Python:

import assemblyai as aai, os

aai.settings.api_key = os.environ["ASSEMBLYAI_API_KEY"]

config = aai.TranscriptionConfig(
    speech_models=["universal-3-5-pro", "universal-2"],  # optional; this is the default
    prompt="Cardiology consultation about chest pain symptoms.",
    keyterms_prompt=["Dr. Smith", "hypertension"],
    speaker_labels=True,
)

transcript = aai.Transcriber(config=config).transcribe("https://assembly.ai/wildfires.mp3")
if transcript.status == aai.TranscriptStatus.error:
    raise RuntimeError(transcript.error)
print(transcript.text)

And in Node/JS:

import { AssemblyAI } from 'assemblyai';

const client = new AssemblyAI({ apiKey: process.env.ASSEMBLYAI_API_KEY });

const transcript = await client.transcripts.transcribe({
  audio: 'https://assembly.ai/wildfires.mp3',
  speech_models: ["universal-3-5-pro", "universal-2"],
  prompt: "Cardiology consultation about chest pain symptoms.",
  keyterms_prompt: ["Dr. Smith", "hypertension"],
  speaker_labels: true,
});
if (transcript.status === 'error') throw new Error(transcript.error);
console.log(transcript.text);

A couple of things worth calling out. The auth header is your raw API key — no Bearer prefix. And speech_models is plural on async: it's an ordered fallback list, defaulting to ["universal-3-5-pro", "universal-2"], so if a language falls outside the flagship's coverage it degrades gracefully to Universal-2's 99+ languages.

Keyterm prompting vs. natural-language prompting

People conflate these two, so let's be precise about what each does.

Keyterm prompting is deterministic. When you pass keyterms_prompt=["Kelly Byrne-Donoghue"], you're telling the model: whenever you hear this, spell it this way. It's the right tool for proper nouns, product names, and anything where a single wrong character is a failed transcript.

Natural-language prompting is probabilistic. When you pass prompt="League of Legends roles", you're biasing the model toward a domain so it resolves ambiguity the way a human familiar with that domain would. It won't force a specific spelling, but it will nudge dozens of borderline decisions in the right direction.

The important, counterintuitive finding: keyterms passed alongside context in the prompt beat keyterms alone. The context tells the model why those terms matter and where they fit, which sharpens recognition well beyond a bare list. That's the whole case for using both fields together.

Does prompting actually improve accuracy?

Yes — and not by a rounding-error margin. Real production audio gets materially more accurate when you describe it and pin the vocabulary, and it works without any fine-tuning: no training run, no custom deployment, just a description of the audio and a list of terms sent at request time. That's the point the market keeps missing — prompting a strong Speech Language Model closes most of the gap people assume requires fine-tuning.

Metaview, which builds interview intelligence for recruiting, is a clear example of contextual prompting in production. As co-founder and CTO Shahriar Tajbakhsh puts it: "Since moving to AssemblyAI, we've seen a meaningful improvement in the confidence tail of our production transcripts....What stands out is not just the model quality, but the way [they] let us bring real meeting context into transcription, from calendar titles to organizations, domains, and participant names, so recruiting conversations come through with the nuance our customers depend on."

If you want to see how we measure this, our guide on how accurate speech-to-text really is and the method behind evaluating speech recognition models both walk through the setup.

prompt vs keyterms_prompt: which should you use?

Short answer — usually both. Longer answer:

• Reach for prompt when the audio has a clear subject the model benefits from knowing: a cardiology consult, a fantasy-sports podcast, a mortgage call. Two to fifty words of plain description.
• Reach for keyterms_prompt when exact spelling is non-negotiable: clinician names, drug names, ticker symbols, internal product names. Up to 1,000 entries, six words max each.
• Use them together whenever both conditions hold — which, in real production audio, is most of the time.

Prompting for specialized domains

Healthcare is the clearest example of the payoff. Pair contextual prompting with Medical Mode — set domain="medical-v1" — and you get a model tuned for clinical language plus your own context on top:

config = aai.TranscriptionConfig(
    speech_models=["universal-3-5-pro"],
    domain="medical-v1",
    prompt="Cardiology consultation about chest pain symptoms.",
    keyterms_prompt=["Dr. Smith", "hypertension", "myocardial infarction"],
)

In our testing, contextual prompting cut the rate of missed medical terms by 31%. The same pattern — describe the scenario, pin the vocabulary — carries over to legal, finance, and any vertical with a dense proper-noun vocabulary.

Async vs streaming: which prompting guide do you need?

This is worth being explicit about, because the tools differ by surface.

• Async / pre-recorded (this guide): you have a file. Use prompt and keyterms_prompt on Universal-3.5 Pro. Full context is available up front because the whole recording exists.
• Streaming / real-time: you have a live conversation. There's no complete file yet, so prompting happens turn by turn with agent_context (seeded at connection time and updated mid-stream) plus rolling Context Carryover. That's a different API and a different mental model — covered in the real-time prompting guide.

If you're not sure which you're building, ask whether you're processing a recording (async) or a live stream (streaming). That answer picks your guide.

Try it yourself

Get your free API key and prompt your first transcript in a few minutes. Want to experiment without writing code first? Test prompting in the playground. Curious how the accuracy stacks up? See the latest benchmarks.

Frequently asked questions

Does prompting actually improve speech-to-text accuracy?

Yes. Passing a natural-language description of the audio plus a list of key terms materially improves accuracy on domain-specific and hard-to-hear audio — with no fine-tuning, since the context is sent at request time. Teams like Metaview rely on exactly this to bring real meeting context into transcription.

Are prompt and keyterms_prompt mutually exclusive?

No. They're separate, complementary tools. prompt describes what the audio is about; keyterms_prompt pins exact spellings. Use both together for the best results.

Why do keyterms work better with a prompt than on their own?

Context tells the model why those terms matter and where they fit, so it resolves ambiguity around them more reliably. Keyterms-with-context consistently beat keyterms alone in our testing.

Which model supports prompting, and what does it cost?

Universal-3.5 Pro, the async flagship, at $0.21/hr. Both prompt and keyterms_prompt are supported on pre-recorded transcription.

How many keyterms can I pass?

Up to 1,000 words or phrases on async, with a maximum of six words per phrase. (Streaming keyterms have a different, smaller cap.)

Can I control spelling and formatting through prompting?

You can control spelling with keyterms_prompt. The prompt field sets context, not formatting — behavioral and formatting instructions there are ignored. Use dedicated formatting params like punctuate and format_text for output shape.