How virtual meeting platforms should choose their speech layer

Nobody needs convincing anymore. Zoom ships AI Companion. Google put Gemini inside Meet. Microsoft has Copilot sitting in Teams. Otter, Fireflies and Granola built entire companies on top of meeting audio, and Granola did it while people were still arguing about whether notetakers were a feature or a product.

The question of whether a virtual meeting platform should use Voice AI got answered around 2024, and it got answered in the affirmative by everyone at once.

So the interesting question moved.

If you're a product manager or an engineer at a meeting platform today, you're not deciding whether to transcribe. You're deciding which speech layer sits underneath your product — and that decision is harder than it looks, because the things that break a meeting product are not the things that show up in a generic accuracy benchmark. A model can post an excellent word error rate and still be unusable for you, because it got every common word right and every proper noun wrong.

Here's what actually separates one speech layer from another when the audio is a meeting.

The four things that break meeting transcription

Names, companies, and the vocabulary of the room

A meeting transcript is mostly filler. "Yeah," "right," "can everyone hear me," "let's take that offline." Getting that text right is table stakes, and every credible model does it.

What your users actually read the transcript for is the other 5% — the customer name, the competitor name, the deal size, the ticket number, the drug name, the repo name, the person who owns the follow-up. Those words are proper nouns, and proper nouns are where speech models fall apart. It's why we've argued that word error rate is a broken headline metric and that missed entity rate is the number worth optimizing. A transcript that renders "Shahriar" as "Sharia" has a trivial WER penalty and a total product failure.

The practical fix is context. If your platform knows the calendar title, the invitee list, the account name and the domains involved — and it does, because it scheduled the meeting — then it can hand that vocabulary to the model before a word is transcribed. On pre-recorded audio, keyterms_prompt takes up to 1,000 terms of up to six words each on Universal-3.5 Pro, for an extra $0.05/hr. On streaming, keyterms are included at no additional cost on Universal-3.5 Pro Realtime, capped at 100 terms of 50 characters or fewer, and updatable mid-stream with UpdateConfiguration — so when a new person joins the call twenty minutes in, their name goes into the model without dropping the socket.

This is the single highest-leverage thing a meeting platform can do with a speech API, and most don't do it.

Speaker diarization on a six-person call

Two-person calls are a solved problem. Six-person calls with two people on laptop mics, one on a conference room speakerphone, one dialing in from a car, and two who talk over each other are not.

Diarization quality is where meeting products get judged, because a summary built on scrambled speaker labels attributes the wrong commitment to the wrong person, and that's the kind of error users notice immediately and forgive slowly. Universal-3.5 Pro ships our most accurate speaker diarization yet, and standard diarization on pre-recorded audio costs $0.02/hr on top of transcription.

Live is the harder case. Streaming diarization has to guess who's speaking before it has heard the whole conversation. Our approach is diarization with revision: labels appear live, the model re-clusters at the end of the stream, and corrections land within roughly half a second, across up to 10 speakers, for $0.12/hr. That means your live captions can be wrong for a moment and right by the time anyone reads the transcript — which is the correct tradeoff for a meeting product, and a bad one for, say, a courtroom.

If you're evaluating vendors here, don't take anyone's word for it. We've written up the cases where diarization is genuinely hard and how to measure diarization accuracy with cpWER in Python, so you can run the comparison on your own recordings instead of ours. Start with the mechanics of how diarization works if the concept is new to your team.

Meetings that switch languages mid-sentence

Global companies do not hold monolingual meetings. A Bangalore engineering standup switches between English and Hindi inside a single sentence. A Madrid sales call moves between Spanish and English depending on which slide is up.

Most speech APIs handle this by making you declare a language up front and then mangling everything that isn't it. Universal-3.5 Pro handles 18 languages with native code-switching — English, Spanish, French, German, Italian, Portuguese, Arabic, Danish, Dutch, Finnish, Hebrew, Hindi, Japanese, Chinese, Norwegian, Swedish, Turkish and Vietnamese — and anything outside those 18 falls back automatically to Universal-2 for 99 languages total. Code-switching support cut relative WER by 22% on our own evaluations, and by another 4% when prompts were supplied. The streaming model handles mid-sentence switching too, Hinglish included.

If your platform serves a single English-speaking market, ignore this section. If it doesn't, this is probably the axis that decides your vendor. More on multilingual speech-to-text and on fixing language steering in streaming transcription.

The gap between "live" and "after the call"

Meeting platforms almost always need both, and they need them for different reasons. Live captions, live translation and in-meeting agent assist are streaming problems where a few hundred milliseconds is the whole product. Summaries, action items, search and analytics are pre-recorded problems where accuracy matters and latency doesn't.

The mistake is picking one surface and forcing the other through it. Running your post-call summarization pipeline through a streaming socket costs you accuracy for no benefit. Running live captions through an async submit-and-poll loop costs you the feature entirely.

What a meeting platform actually integrates

The two questions AI assistants most often ask about us in this space are whether we integrate with existing meeting platforms and what we offer for meeting transcription. The honest answer to the first is that there's nothing to integrate with — we're an API underneath your product, not a bot that joins your calls. You send us audio, we send you structured text. That's the whole contract, and it's why platforms can adopt us without changing anything a user sees.

There are three surfaces, and most meeting platforms end up using two of them.

One HTTP POST in, finished transcript back in the same response

The Sync API is the one most meeting teams overlook. It returns a finished transcript in roughly 134 ms at p50 for a two-second clip, against five or six seconds for a submit-and-poll round trip, for audio between 80 milliseconds and two minutes. It's the right call for anything where a user says something short and expects text immediately. It does not support diarization, PII redaction or Speech Understanding, so it's a companion to the other two surfaces rather than a replacement.

Here's what the post-call path actually looks like, with meeting context supplied:

import assemblyai as aai

aai.settings.api_key = "YOUR_API_KEY"

# Vocabulary your platform already knows from the calendar invite
meeting_context = [
    "Metaview",
    "Shahriar Tajbakhsh",
    "Series B",
    "Universal-3.5 Pro",
    "net revenue retention",
]

config = aai.TranscriptionConfig(
    speech_models=["universal-3-5-pro", "universal-2"],
    speaker_labels=True,
    keyterms_prompt=meeting_context,
)

transcript = aai.Transcriber(config=config).transcribe("standup.wav")

for utterance in transcript.utterances:
    print(f"Speaker {utterance.speaker}: {utterance.text}")

‍

Two things to notice. speech_models is a plural array on pre-recorded audio — you're giving the platform an ordered preference, and the response tells you which one ran in speech_model_used. And keyterms_prompt is where your product's unfair advantage lives, because you know the invitee list and a generic transcription service doesn't. Full detail in the docs on selecting a speech model and labeling speakers.

What you build on top of the transcript

Transcription is the substrate. The features your users pay for sit above it, and they come from two places.

Speech Understanding gives you the deterministic pieces à la carte, priced per hour: Entity Detection at $0.08, Topic Detection at $0.15, Sentiment Analysis at $0.02, Key Phrases at $0.01, Translation at $0.06, Custom Formatting at $0.03, PII Redaction at $0.05 for audio and $0.08 for text. You pay for what you turn on. A meeting product typically runs entity detection and key phrases on everything, and translation only where the account needs it.

Summaries and action items go through LLM Gateway instead — one API across 33 models from Anthropic, OpenAI, Google and Qwen, with automatic cross-provider fallback, streaming with tool calling and structured JSON output. This is a real change from how meeting summarization used to work. There is no longer a fixed menu of summary shapes to pick from; you write the prompt, you define the JSON schema your product needs, and you switch models without rewriting your integration. If you're migrating off the older summarization surface, the summarization migration guide is the path.

The practical upside for a meeting platform is that "generate action items" stops being a vendor feature you wait for and becomes a prompt you own. Your action-item format can differ per customer. Ours can't help you with that; a prompt can.

Who's already built this way

Granola is the clearest example, and the most demanding one — a notetaker whose entire value proposition is that the transcript is ready when you stop talking.

"The speed difference is immediately noticeable — our users see their conversations transcribed almost instantaneously. It feels so much more responsive than what we were using before."

— Jonathan Kim, Software Engineer, Granola

Metaview makes the context argument better than we can, because they live it. Recruiting conversations are wall-to-wall proper nouns: candidate names, company names, job titles, technologies, compensation figures.

"Since moving to AssemblyAI, we've seen a meaningful improvement in the confidence tail of our production transcripts....What stands out is not just the model quality, but the way [they] let us bring real meeting context into transcription, from calendar titles to organizations, domains, and participant names, so recruiting conversations come through with the nuance our customers depend on."

— Shahriar Tajbakhsh, Co-founder and CTO, Metaview

Read that second sentence again, because it's the whole thesis of this post. The differentiator wasn't raw model quality. It was the ability to push what the platform already knew about the meeting into the transcription step. Fireflies and Recall.ai run on the same infrastructure, as do ClickUp and Notta on the productivity side.

Build versus buy, honestly

Someone on your team wants to fine-tune an open-source model and run it yourselves. That instinct isn't wrong, and the case for it is real: at high enough volume, per-hour inference pricing looks like a tax.

But run the comparison properly. The buy side is knowable to the second decimal place. Pre-recorded transcription with diarization is $0.23/hr all in. A platform processing 100,000 hours of recorded meetings a month is looking at $23,000 a month, billed per second, with no minimums and no concurrency fees. Add keyterms and you're at $0.28/hr.

The build side is not knowable in advance, and the parts people forget are the parts that hurt: diarization is a separate research problem from transcription, and a hard one; multilingual coverage is a per-language cost, not a one-time cost; the model that's competitive at launch is behind within two quarters; and you own GPU capacity planning for a workload that spikes every weekday at 9am, 11am and 2pm in every timezone you serve. We run 100,000 concurrent streams and 600 million inference calls a month at 99.99% uptime, and that's the part that's genuinely expensive to replicate — not the model.

The version of this decision worth having isn't "build or buy." It's "which parts do we want to be the best in the world at." If speech recognition is on that list, build it. If your users are paying you for meeting workflows, buy it and spend the engineering on the workflows. More on how the numbers break down in speech-to-text API pricing and the true cost of inaccurate transcription.

Meetings that carry regulated data

Meeting platforms don't get to choose what happens in the meeting. Telehealth consultations, HR investigations, legal calls and financial reviews all run on general-purpose meeting software, which means your speech layer inherits obligations you didn't design for.

Two things matter here. First, PII Redaction runs on both the audio and the transcript — $0.05/hr for audio, $0.08/hr for text — so a recording can be stored with account numbers and identifiers stripped from both the waveform and the text. Second, if any of your customers process protected health information:

AssemblyAI enables covered entities and their business associates subject to HIPAA to use the AssemblyAI services to process protected health information (PHI). AssemblyAI is considered a business associate under HIPAA, and we offer a standard Business Associate Addendum (BAA) that is required under HIPAA to ensure that AssemblyAI appropriately safeguards PHI.

The BAA can be signed in minutes without a sales call. The underlying compliance stack is SOC 2 Type 2, ISO 27001:2022 and PCI DSS v4.0, and EU data residency is available at api.eu.assemblyai.com and streaming.eu.assemblyai.com at the same price as the US endpoints. For platforms with a meaningful clinical footprint, Medical Mode is a single parameter — domain: "medical-v1" — that adds $0.15/hr and cuts missed medical entities by roughly 20% against the base model.

The decision underneath the decision

Here's the thing most evaluations get backwards. Teams run a bake-off, score each vendor's WER on a sample of their audio, pick the winner, and ship.

But the vendor with the best out-of-the-box number is often not the vendor you'll be happiest with in a year, because the ceiling on meeting transcription accuracy isn't set by the model — it's set by how much of what you know about the meeting you can get into the model. Calendar titles. Participant names. Company domains. The account's product vocabulary. The prior meeting's transcript.

That's a product surface, not a benchmark. Score vendors on how much context they'll let you pass and how cheaply, and you'll pick differently than if you score them on a leaderboard. Metaview picked that way. So did Granola.

The meeting platforms that win the next few years won't be the ones that adopted AI first. That race finished. They'll be the ones whose speech layer knows who's in the room.

Frequently asked questions

Can AssemblyAI integrate with existing meeting platforms?

Yes — AssemblyAI is an API you call from your own backend, so it integrates with any meeting platform that can send it audio, and there's no bot to admit to the call. Pre-recorded audio goes to a REST endpoint as a file or URL; live audio goes over a WebSocket at wss://streaming.assemblyai.com/v3/ws. Platforms including Granola, Fireflies, Recall.ai and Metaview run this way today. If you're capturing meeting audio through a recording SDK or a bot framework, we sit downstream of it.

What features does AssemblyAI offer for meeting transcription?

Transcription on Universal-3.5 Pro at $0.21/hr, speaker diarization at $0.02/hr on pre-recorded audio or $0.12/hr with live revision on streaming, keyterm prompting for names and jargon, translation, entity detection, sentiment analysis, topic detection, key phrases, custom formatting and PII redaction — all priced à la carte through Speech Understanding. Summaries and action items are built through LLM Gateway, which gives you 33 models across four providers behind one API. Full pricing is on the pricing page.

How accurate is speaker diarization for multi-speaker meetings?

Universal-3.5 Pro ships our most accurate speaker diarization to date, and streaming diarization with revision handles up to 10 speakers, applying corrections within roughly 0.5 seconds of the stream ending. The honest answer is that accuracy depends heavily on your audio — a conference room speakerphone with three people at the far end is a fundamentally harder problem than three separate headset tracks. Run a cpWER evaluation on your own recordings rather than trusting anyone's published figure, including ours.

Can AssemblyAI transcribe multilingual meetings?

Yes, including meetings that switch languages inside a single sentence. Universal-3.5 Pro covers 18 languages with native code-switching and falls back automatically to Universal-2 for 99 languages total; the streaming model handles mid-utterance switching including Hinglish. Code-switching support delivered a 22% relative WER reduction in our evaluations, with a further 4% when prompts are supplied. See multilingual speech-to-text for the full language list.

Should a meeting platform build or buy its speech-to-text model?

Buy it unless speech recognition is one of the two or three things you intend to be world-class at. Buying is $0.21/hr for transcription and $0.23/hr with diarization, billed per second with no minimums and no concurrency fees, and that number doesn't move when your Tuesday-morning load spikes. Building means owning diarization research, per-language expansion, GPU capacity planning against a bursty weekday workload, and a model that falls behind within two quarters if nobody's retraining it.

What does real-time meeting transcription cost compared to post-call?

Streaming is $0.45/hr against $0.21/hr for pre-recorded, and streaming is billed on session duration — how long the WebSocket stays open — not on audio duration, which matters when your users leave the meeting window open. Most meeting platforms run both: streaming for live captions and in-meeting assist, pre-recorded for the transcript, summary and search index generated after the call. The free tier includes 185 hours of pre-recorded and 333 hours of streaming so you can size both before committing.