New AssemblyAI-TTS-1: Text-to-speech for voice agents. Learn more
Insights & Use Cases

How to choose the best speech-to-text API

With more speech-to-text APIs on the market than ever before, how do you choose the best one for your product or use case? Answering these six questions is a great starting point.

Abstract green cone illustration

Written by

Kelsey Foster

Published on

26 October 2026

The best speech-to-text API is the one that wins on your own audio. For most production teams in 2026, start with AssemblyAI: Universal-3.5 Pro for pre-recorded audio ($0.21/hr) and Universal-3.6 Pro Realtime for streaming ($0.45/hr). Then benchmark two or three alternatives on word error rate, entity accuracy, latency, and total cost.

If you're building…Start withWhy (dated proof)
Pre-recorded transcription, meeting notes, analyticsAssemblyAI Universal-3.5 Pro4.35% average WER and 30.17% average cpWER, lowest on AssemblyAI's published benchmarks (accessed October 2026)
Voice agents, live captions, agent assistAssemblyAI Universal-3.6 Pro Realtime5.19% WER and 14.4% entity error rate on 12,460 voice-agent scenarios (published September 29, 2026)
A full voice agent without stitching vendorsAssemblyAI Universal-3.6 Pro Realtime + Text-to-Speech APIBoth speech ends from one vendor, orchestrated with LiveKit, Pipecat, or your own loop
99+ languages on a budgetAssemblyAI Universal-2Widest language coverage at $0.15/hr
Live dictationAssemblyAI Dictation API134 ms p50 latency, 19 languages, $0.62/hr (launched September 15, 2026)

That's the short answer. The rest of this guide shows how to check it for your workload.

Most teams pick a speech-to-text API the same way: run a few clean sample files, compare the word error rate, check the price page, and ship. Then production audio shows up. Callers on speakerphone, two people talking over each other, a customer who switches from English to Spanish mid-sentence, a product name the model has never heard.

That's where providers start to differ.

This guide gives you eight criteria, a scorecard you can weight for your use case, a dated comparison of AssemblyAI against Deepgram, ElevenLabs, OpenAI, and others, a shortlist by use case, and a benchmarking method that holds up when you put real files through it.

Build it

Pick the right model for your audio, then transcribe your first file to test it on your own recordings.

Building with a coding agent? Connect our docs MCP server with claude mcp add assemblyai-docs --transport http https://assemblyai.com/docs/mcp, or install the skill with npx skills add AssemblyAI/assemblyai-skill.

Open model selection

What is a speech-to-text API?

A speech-to-text API is a developer interface that converts spoken audio into written text. You send an audio file or a live stream to an endpoint and get back a transcript, usually with word-level timestamps, speaker labels, and confidence scores. The provider runs the models and the infrastructure, so your team doesn't have to train or host anything.

There are three ways to deploy one. Asynchronous (batch) APIs process pre-recorded files. Streaming APIs transcribe live audio over a persistent WebSocket connection. Self-hosted deployments run the models inside your own environment for strict data-control requirements. Batch and streaming models have different accuracy profiles and prices, so evaluate them separately.

The eight criteria for choosing a speech-to-text API

CriterionWhat to checkWhy it matters
AccuracyError rate on your audio, entity accuracy (names, numbers, emails), hallucinations on silenceTranscript errors carry into summaries, analytics, and agent responses
Latency and real-timeStreaming support, endpointing behavior, time to first and final transcriptDecides whether live captions and voice agents feel responsive
Language coverageSupported languages, code-switching, automatic language detectionGlobal products rarely get one language per file
PricingPer-hour rates, add-on costs, minimums, contractsList price is only part of total cost
Scale and concurrencyConcurrency limits, status history, volume processedA great demo means nothing if the API caps out under load
ComplianceSOC 2 Type 2, BAA availability, EU data residency, retention controlsRequired for regulated industries and most enterprise security reviews
Developer experienceDocs, SDKs, time from API key to first transcriptIntegration effort compounds over the life of the product
Support and innovationSupport channels, changelog frequency, model release cadenceVoice AI models improve quickly, and stagnant providers fall behind

Speech-to-text API scorecard: weight the 8 criteria for your use case

Not every criterion matters equally. A voice agent lives or dies on latency and entity accuracy. A medical scribe can tolerate a few extra seconds but not a missed medication name. So weight first, then score.

Copy the table below, adjust the weights so each column adds up to 100, then score each provider from 1 to 5 on every criterion using your own benchmark results. Multiply score by weight and add it up.

CriterionAI notetakerContact centerVoice agentMedical
Accuracy25202030
Latency and real-time515255
Language coverage1515105
Pricing15151010
Scale and concurrency1015105
Compliance1010530
Developer experience105155
Support and innovation105510

These weights are a starting point, not a rule. If your notetaker serves a global sales team, push language coverage up. If your contact center handles card payments, compliance moves higher.

Score AssemblyAI on your own audio

Run your benchmark files through Universal-3.5 Pro and Universal-3.6 Pro Realtime in minutes. Free to start, pay-as-you-go after that, no contract.

Sign up free

Accuracy: how to measure it without fooling yourself

Word error rate (WER) is the standard metric. It counts the words a model inserted, deleted, or substituted compared with a human reference transcript, divided by the number of words in the reference.

It's useful. It's also easy to misread.

Modern models now catch words that human transcribers miss, which shows up as "insertions" against a flawed reference. And WER weighs every word the same, so "the" counts as much as a patient's medication or a caller's account number. We wrote more about this in word error rate is broken. Track these alongside WER:

  • Entity error rate: how often the model gets names, phone numbers, emails, dates, and IDs wrong. These are the words your downstream systems act on.
  • cpWER for multi-speaker audio: concatenated minimum-permutation WER scores the transcript and speaker attribution together, so a correct sentence assigned to the wrong speaker still counts as an error. Here's why cpWER beats DER.
  • Hallucinations on silence: what the model returns when nobody is speaking, such as hold music or line noise.

How to run a benchmark you can trust

  1. Collect 10 to 20 files that represent your production audio, including the hard ones: noisy, accented, multi-speaker, domain-heavy.
  2. Get human-verified reference transcripts for each file.
  3. Send the identical files to every provider with comparable settings.
  4. Normalize the outputs (casing, punctuation, numbers) before scoring.
  5. Calculate WER and entity error rate per file, not just the average.
  6. Listen to flagged insertions before counting them as errors. The reference is sometimes the one that's wrong.

For metric definitions, normalization code, and how to automate this as a regression test, see how to measure transcription accuracy.

How AssemblyAI compares: dated, sourced benchmarks

Every number below names its source and date. Treat them as a reason to shortlist, not a reason to skip your own test. For the full model-by-model breakdown, see how accurate speech-to-text is in 2026.

Pre-recorded audio

On AssemblyAI's published benchmarks (accessed October 8, 2026), Universal-3.5 Pro averaged 4.35% WER across four dataset groups: synthetic medical, accented English (India), general speech, and webinar speech.

ModelAverage WER, 4 dataset groups (lower is better)
AssemblyAI Universal-3.5 Pro4.35%
Mistral Voxtral Mini5.24%
OpenAI GPT-4o Transcribe5.34%
ElevenLabs Scribe v25.87%
Gladia6.47%
Deepgram Nova-36.66%
Azure Batch7.02%

Code-switching is where the spread gets wide. On a five-language-pair code-switching benchmark published with the Universal-3.5 Pro launch (July 7, 2026), Universal-3.5 Pro averaged 7.69% normalized WER, versus 8.77% for ElevenLabs Scribe v2, 12.22% for Deepgram Nova-3 Multilingual, and 44.58% for OpenAI GPT-4o Transcribe.

For diarization, Universal-3.5 Pro posted the lowest average cpWER on the benchmarks page: 30.17% across DiPCo, CALLHOME, NOTSOFAR, and AMI. Azure was close behind at 30.35%, followed by ElevenLabs Scribe v2 (35.26%), Speechmatics (36.60%), Gladia (36.88%), and Deepgram (37.93%).

Streaming audio

Universal-3.6 Pro Realtime was tested on 12,460 scripted caller scenarios with real speakers across three noise environments (published September 29, 2026; benchmark data as of September 16, 2026):

Metric (lower is better)Universal-3.6 Pro RealtimeElevenLabs Scribe v2Deepgram Nova-3Deepgram Flux EN
Word error rate5.19%7.78%8.64%13.50%
Entity error rate14.4%18.5%26.1%30.1%
WER, heavy noise6.13%9.64%9.93%14.85%
Code-switching WER, 5 pairs7.20%13.01%17.13%52.86%

A third-party leaderboard points the same way. On Pipecat's open STT benchmark (rendered September 28, 2026), Universal-3.6 Pro Realtime posted 0.96% pooled semantic WER at 307 ms median time to first speech.

Test accuracy on your own files

Upload your hardest recordings and compare transcripts side by side. No code required.

Try playground

Latency and real-time capabilities

Streaming and batch solve different problems. Streaming APIs return partial and final transcripts over a WebSocket as someone speaks. Batch APIs return a full transcript after the file is processed. Don't compare them head to head.

Providers also measure latency differently. Some report time to first word, others time to a complete utterance. Test in your own environment with your audio and your configuration.

For voice agents, the number that matters most is often endpointing: how fast and how reliably the model decides the speaker is done. Cut someone off and the agent feels rude. Wait too long and it feels slow. Universal-3.6 Pro Realtime raised end-of-turn precision from 74.1% to 77.9% over Universal-3.5 Pro Realtime, with median endpoint latency unchanged at 537 ms. It also cut errors on short utterances (the one-word answers and spelled-out confirmations agents rely on) from 2.65% to 1.45%.

Silence handling is the other test most teams skip. On 2,950 clips with no speech, such as hold music, line noise, and silence, Universal-3.6 Pro Realtime returned text 0.27% of the time. Deepgram Flux EN returned text on 38.75%.

Other features to look for in a streaming model include entity-aware endpointing, voice focus to suppress background speakers, automatic language detection with optional pinning, and an end-of-turn confidence signal your agent logic can act on. Universal-3.6 Pro Realtime ships all four. For a voice-agent-specific checklist, see how to choose a speech-to-text API for voice agents.

Customer feedback lines up with the benchmark story:

"We were searching for the best realtime ASR model for our voice agent pipeline in Fireflies. The new Universal 3.5 Pro speech model from Assembly is best so far in terms of accuracy, latency and language switching."

— Foysal Osmany, Software Engineer at Fireflies

Language coverage

Count the languages, then check how the model handles them. A supported language and a well-supported language aren't the same thing.

  • Universal-3.5 Pro (pre-recorded): 18 languages with native code-switching, no configuration needed. Every word is transcribed in the language it was spoken.
  • Universal-3.6 Pro Realtime (streaming): 32 languages with automatic detection, including 14 new additions such as Korean, Russian, Cantonese, and Urdu.
  • Universal-2 (pre-recorded): 99+ languages for the widest coverage.

If your users switch languages mid-conversation, test code-switching directly. Many models treat it as an edge case.

Pricing and total cost of ownership

Per-hour rates are the easiest number to compare and the least complete. Total cost includes add-ons, integration time, and the cost of fixing errors downstream: a missed account number means a failed lookup, a wrong drug name means a clinician correcting the note. A cheaper model that needs more human review often costs more.

Look for transparent pricing with no minimums or forced contracts. AssemblyAI's public pay-as-you-go rates:

Model or productTypePrice
Universal-3.5 ProPre-recorded$0.21/hr
Universal-2Pre-recorded, 99+ languages$0.15/hr
Universal-3.6 Pro RealtimeStreaming$0.45/hr base, volume discounts at any tier
Dictation APILive dictation$0.62/hr

Pre-recorded add-ons are priced separately: keyterms prompting is +$0.05/hr, standard diarization +$0.02/hr, and Medical Mode +$0.15/hr. EU data residency costs the same as US processing. For a side-by-side of provider rates, see our speech-to-text API pricing comparison.

Scale and concurrency

Ask how much audio the provider processes today, what its concurrency limits are, and what its status history looks like. A managed API should scale as your request volume grows without capacity planning on your side. AssemblyAI offers concurrency that scales with your account on pay-as-you-go pricing, with no spend commitment to unlock it (paid plans start at 200 concurrent jobs).

Compliance and data security

Start with encryption in transit and at rest, SOC 2 Type 2, and GDPR alignment. Then check the specifics your industry needs:

  • Healthcare: a signed Business Associate Addendum (BAA). AssemblyAI is considered a business associate under HIPAA and offers a standard BAA, which paid accounts can review and sign in the dashboard without a sales call. Details are in the BAA FAQ.
  • Data residency: EU processing and storage for customers who need data to stay in the EU.
  • Retention controls: options to limit how long audio and transcripts are stored, and to opt out of model training.

If you process PHI, ask every vendor on your shortlist whether they'll sign a BAA before you run a single test. For a deeper enterprise checklist, see what to evaluate in an enterprise speech-to-text API.

Developer experience, support, and innovation

Good docs and SDKs in your language should get you from API key to first transcript in minutes. If that takes a day, integration won't get easier from there.

Support is harder to judge from the outside. Ask what channels you get (email, chat, a shared Slack channel) and whether production teams get dedicated engineers. Larger teams benefit when a provider embeds engineers who help tune performance at scale.

For innovation, the public changelog is the clearest signal. Frequent, detailed release notes mean the provider is shipping. A stale changelog is a red flag in a field where models improve every few months.

Also check which Speech Understanding features come with the API, such as PII redaction, entity detection, sentiment analysis, and summarization through an LLM gateway, and how they're priced.

Shortlist by use case

If you want a broader list of providers, our roundup of the best speech-to-text APIs for startups compares them in one table. Here's how the criteria play out for four common workloads.

AI notetakers and meeting tools

Prioritize diarization accuracy (cpWER), code-switching, and entity accuracy on names and company terms. Universal-3.5 Pro is built for this: native code-switching across 18 languages, the lowest average cpWER on AssemblyAI's published benchmarks, and contextual prompting so you can pass in meeting titles or participant names.

Contact centers

Prioritize accuracy under noise, entity accuracy on phone numbers and account IDs, and scale. Use Universal-3.6 Pro Realtime for live agent assist and Universal-3.5 Pro for post-call analytics. Pair them with PII redaction for calls that include payment or personal data.

Voice agents

Prioritize endpointing, short-utterance accuracy, and silence handling. Universal-3.6 Pro Realtime is the streaming model to benchmark. If you'd rather not stitch together separate speech-to-text and text-to-speech vendors, pair it with AssemblyAI's Text-to-Speech API for voice output and run the loop with LiveKit, Pipecat, or your own code.

Medical transcription

Prioritize medical entity accuracy and a signed BAA. Universal-3.5 Pro with Medical Mode (one parameter: domain: "medical-v1") cuts missed medical entities by about 20% compared with the base model. Medical Mode supports English, Spanish, German, and French for both pre-recorded and streaming audio. It works with all of AssemblyAI's real-time models, including Universal-3.6 Pro Realtime, and costs $0.15/hr on top of the base rate ($0.36/hr combined with Universal-3.5 Pro).

Contextual prompting helps here too. In internal healthcare testing, feeding the model a patient's prior-visit note cut missed medical terms by 31%. AssemblyAI offers a BAA for customers processing PHI.

Matching AssemblyAI models to your workload

Use caseRecommended modelWhy
Pre-recorded or batch transcriptionUniversal-3.5 ProNative code-switching across 18 languages, lowest published cpWER, contextual prompting, $0.21/hr
Pre-recorded, 99+ languagesUniversal-2Widest language coverage at $0.15/hr
Real-time transcription, voice agents, live agent assistUniversal-3.6 Pro Realtime5.19% WER and 14.4% entity error rate on voice-agent audio, 32 languages, $0.45/hr
Live dictationDictation API134 ms p50 latency, 19 languages, $0.62/hr
Medical transcriptionUniversal-3.5 Pro + Medical Mode~20% fewer missed medical entities than the base model, BAA available, $0.36/hr combined
Full voice agent stackUniversal-3.6 Pro Realtime + Text-to-Speech APIBoth speech ends from one vendor, orchestrated with LiveKit, Pipecat, or your own loop

Choosing the right speech-to-text API: the short version

Weight the eight criteria for your use case. Shortlist two or three providers. Benchmark them on your own audio, including the files you'd rather not show anyone.

One more thing to plan for: the model you choose today won't be the model you run in a year. AssemblyAI shipped Universal-3.5 Pro in July and Universal-3.6 Pro Realtime in September 2026. So build your benchmark set once, keep it, and rerun it every time a provider ships. That turns vendor selection from a one-time bet into a repeatable test, and it's the fastest way to know when switching or upgrading is worth it.

Start your benchmark today

Get a free API key, run your test set through Universal-3.5 Pro and Universal-3.6 Pro Realtime, and see the results on your own audio.

Sign up free

Frequently asked questions

What is the best speech-to-text API in 2026?

For most production teams, AssemblyAI is the best place to start: Universal-3.5 Pro for pre-recorded audio and Universal-3.6 Pro Realtime for streaming. Universal-3.5 Pro posted the lowest average WER (4.35%) and cpWER (30.17%) on AssemblyAI's published benchmarks, and Universal-3.6 Pro Realtime posted 5.19% WER on 12,460 voice-agent scenarios. There's no single best API for every product, so benchmark your finalists on your own files.

What is the most accurate speech-to-text API?

Accuracy depends on the audio, so the most accurate API for clean podcasts may not be the most accurate for noisy phone calls. On published benchmarks, AssemblyAI's Universal-3.6 Pro Realtime posted 5.19% WER and 14.4% entity error rate on voice-agent audio, ahead of ElevenLabs Scribe v2 (7.78%, 18.5%) and Deepgram Nova-3 (8.64%, 26.1%). Run your own audio through your shortlist before deciding.

Which speech-to-text API is best for voice agents?

For voice agents, AssemblyAI's Universal-3.6 Pro Realtime is the streaming model to benchmark first. It cut short-utterance errors to 1.45%, returned text on only 0.27% of no-speech clips, and supports 32 languages at $0.45/hr. If you want speech-to-text and text-to-speech from one vendor, pair it with AssemblyAI's Text-to-Speech API for voice output.

How much does a speech-to-text API cost?

Most speech-to-text APIs charge per hour of audio. AssemblyAI's Universal-2 costs $0.15/hr and Universal-3.5 Pro costs $0.21/hr for pre-recorded audio, and Universal-3.6 Pro Realtime costs $0.45/hr for streaming, with no minimums. The lowest list price is rarely the cheapest option once you count add-ons and the cost of fixing errors.

Is there a free speech-to-text API?

Yes. AssemblyAI offers a free tier with no credit card required, and Google Cloud Speech-to-Text includes 60 free minutes per month on its V1 API, according to Google's pricing page. For open-source options, see our guide to free speech-to-text APIs and open-source engines.

What's the difference between real-time and asynchronous transcription?

Asynchronous (batch) transcription processes a complete pre-recorded file and returns the full transcript. Real-time (streaming) transcription converts live audio to text as it's spoken, returning partial and final results over a WebSocket. At AssemblyAI, Universal-3.5 Pro handles pre-recorded audio at $0.21/hr and Universal-3.6 Pro Realtime handles streaming at $0.45/hr. Benchmark them separately.

How do I benchmark speech-to-text APIs on my own audio?

Collect 10 to 20 representative files, get human-verified reference transcripts, send identical files to each provider, normalize the outputs, and compare WER and entity error rate per file. AssemblyAI's guide to measuring transcription accuracy includes normalization code and a CI setup for rerunning the test whenever a model changes.