Best LLM for Summarizing Transcripts: From 30-Second Voicemails to Hour-Long Meetings

“Best LLM for summarization” has four answers, because a 30-second voicemail and a 60-minute meeting are not the same task. The model that nails a two-line voicemail summary in a fraction of a second can lose track of the third agenda item an hour in. The model that handles the hour can be the wrong choice for a voicemail on both speed and price.

This post organizes the decision by transcript length, using Voice Standard Bench, which scores every model on LLM Gateway on summarization at four lengths, on real speech-to-text output rather than clean documents. The picks below are as of October 28, 2026. The live page is the source of truth, and it changes as models change.

Why “best LLM for summarization” has four answers, not one

Most summarization advice online is about documents: ROUGE scores, map-reduce for long PDFs, abstractive versus extractive. Transcripts break those assumptions in two ways. The input is spoken (disfluent, repetitive, two or more speakers, sometimes misrecognized), and the length varies by two orders of magnitude within the same product.

Here’s what changes at each bucket the bench tests:

• 30 seconds (voicemail): one speaker, one purpose. The summary is a few sentences at most. Almost any model gets the content right (41 of the 52 models on the Summarization tab score 90 or higher); the differences are speed, price, and whether the model pads the answer.
• 2 minutes (short call): two speakers, a request, maybe a resolution. Faithfulness starts to matter: did the summary say the refund was approved when the agent only said it was submitted?
• 15 minutes (support call): a problem, several attempts, a resolution, next steps. Coverage separates models. Missing the workaround the agent gave at minute 11 is a real failure. Only 10 models score 90 or higher here.
• 60 minutes (meeting): many topics, decisions, owners, and tangents. The spread between the top and bottom of the ranking is widest here, and the speed gap between small and large models narrows because everyone is processing a lot of tokens.

The bench’s launch post summarizes the overall pattern: most voice tasks don’t need frontier models, or frontier-model prices. Open and small models are competitive on short inputs; the gap opens at meeting length. That’s why the results page has a length picker, and why you should set it before reading any ranking.

How Voice Standard Bench scores summarization

Voice Standard Bench is open source and runs every Gateway model (50+ at the time of writing) on four tasks: summarization, information extraction, text correction, and dictation cleanup. For summarization, every model gets the same prompt: write a summary a busy person could read in place of the call, stating only what was said. It asks for plain prose, not bullet points, unless the call covers more than three distinct topics. It asks the model to cover the topics discussed, then decisions, then action items, and to scale the length to the call: a few sentences for a short call, a short paragraph for a long one, with no padding.

Each output is scored by two LLM judges (Claude Opus 5.5 and GPT-5.6 Sol, tracked separately and then combined) on three criteria:

• Faithfulness: does the summary say only things the transcript supports?
• Coverage: does it include the things that mattered?
• Conciseness: does it stop when it’s done?

Why judges instead of ROUGE or BERTScore? Those metrics compare a summary to a reference summary by token overlap. A summary can paraphrase perfectly and score poorly, or copy the reference’s phrasing while inverting a fact and score well. For spoken input, where the “right” summary is a judgment call about what mattered on the call, a rubric-based judge tracks what a human reviewer would say far better. The methodology post covers judge calibration: the two judges disagree by a few points on average, self-preference bias was measured and found small, and scores within 2 to 3 points of each other should be read as a tie.

Each bucket uses 10 real transcripts within 15% of the target length, produced by AssemblyAI Speech-to-Text, so the input has the disfluencies and occasional misrecognitions a real pipeline sees. Alongside quality, every model gets a median response time and a cost per hour of audio. Requests that error out (timeouts, provider errors) are recorded but left out of the score rather than counted as zero, so a model with many errors can post a high score on the calls that completed.

Cost per hour of audio, not per token

Providers price in tokens. Voice products run in hours of audio. The bench converts one into the other so you can budget on the unit you actually sell: for a given model, task, and length, what does one hour of audio cost to summarize at provider list price?

This matters because the conversion isn’t obvious. Input tokens scale with transcript length, output tokens scale with how verbose the model is, and reasoning models can spend many hidden tokens before writing a word. Two models can differ by orders of magnitude in cost per hour. On 2-minute calls, the top scorer, claude-opus-5-5 (score 99), costs about $0.30 per hour of audio; the Lowest cost pick, nemotron-3-nano-30b-a3b (score 82), costs about $0.0016. That’s a 17-point quality gap at under 1% of the price. Whether those 17 points are worth it is the decision the bench lets you make with numbers.

A quick way to use it: multiply the cost-per-hour figure on the results page by your monthly audio hours. For a support team summarizing 10,000 hours a month, the gap between the top performer and a budget pick is a line item, not a rounding error. The choose a model docs show how to filter the Gateway catalog by task and length to pull the cost figure for a shortlist.

Picks by transcript length

On the live results page, open the Summarization tab and set the length. Each view shows three cards: Highest Performance (the top score; ties go to the lower cost), Lowest cost (the lowest cost per hour of audio), and Fastest (the lowest median response time at that length). Lowest cost and Fastest are raw picks with no quality floor, so check the score before you ship one. If you want picks with a floor, the take-home tables in the launch post list the cheapest and fastest models within 8 points of the top scorer; the live cards don’t apply that rule. Here is each card with its score, as of October 28, 2026:

30-second voicemail

Highest Performance: gpt-5.6-terra (score 100) · Lowest cost: nemotron-3-nano-30b-a3b (score 76, $0.0016 per hour of audio) · Fastest: qwen3.5-4b-32k-fast (score 76, 0.28 s median)

When to choose which: if the summary shows in a UI the moment the voicemail lands, response time is what users feel. The Fastest pick answers in 0.28 seconds but trails the top by 24 points, so look one or two rows down the speed ranking before you take it: most models score in the 90s at this length.

2-minute call

Highest Performance: claude-opus-5-5 (score 99) · Lowest cost: nemotron-3-nano-30b-a3b (score 82, $0.0016 per hour of audio) · Fastest: qwen3.5-4b-32k-fast (score 69, 0.60 s median)

When to choose which: for after-call notes written into a CRM, a low-cost model is usually the right default. Faithfulness matters more than at 30 seconds, and the Lowest cost pick sits 17 points below the top, so check that the score you pick is close enough to the top for your use before committing.

15-minute support call

Highest Performance: claude-opus-5-5 (score 97) · Lowest cost: nemotron-3-nano-30b-a3b (score 77, $0.0013 per hour of audio) · Fastest: qwen3.5-4b-32k-fast (score 60, 1.49 s median)

When to choose which: coverage separates models here. If the summary drives a follow-up (a ticket, an escalation), lean toward Highest Performance and accept the latency; batch processing hides it.

60-minute meeting

Highest Performance: deepseek-v4.1-flash (score 96)* · Lowest cost: nemotron-3-nano-30b-a3b (score 69, $0.0011 per hour of audio) · Fastest: gemini-3.5-flash-lite (score 76, 2.86 s median)

*deepseek-v4.1-flash posts the top 60-minute score at about $0.0092 per hour of audio, but the bench logged 25 errored requests for it across its 60-minute runs. Errored requests are left out of the score, so 96 reflects the calls that completed. Its median response time at this length is 73.3 seconds. The next-highest scores are claude-opus-5-5 at 95 and gpt-6-astra at 95.

When to choose which: this is where model choice matters most. Read the performance spread, not just the cards; if a low-cost model scores near the top, it’s a strong choice because hour-long inputs are where cost per hour adds up fastest. If you need reliability at this length, weigh error behavior on your own long calls as well as the score.

Summarize a transcript through LLM Gateway

LLM Gateway is OpenAI-compatible: same API key as Speech-to-Text, provider list price with 0% markup, and switching models means changing one string. That’s what makes it practical to run the bench’s shortlist on your own calls. The sample below transcribes a file, then sends the transcript to three candidate model IDs with the bench’s own summarization prompt and prints each result with its response time.

Comparing finished summarizer apps instead of models? Our roundup of the best AI transcript summarizers covers the off-the-shelf tools.

import os, time, requests
import assemblyai as aai

API_KEY = os.environ["ASSEMBLYAI_API_KEY"]
aai.settings.api_key = API_KEY

# 1. Transcribe with speaker labels so the summary can attribute statements
config = aai.TranscriptionConfig(speech_models=["universal-3-5-pro"], speaker_labels=True)
transcript = aai.Transcriber().transcribe("support-call.mp3", config=config)
if transcript.status == aai.TranscriptStatus.error:
    raise RuntimeError(transcript.error)
turns = "\n".join(f"Speaker {u.speaker}: {u.text}" for u in transcript.utterances)

# 2. Send the same transcript to candidate models on LLM Gateway
URL = "https://llm-gateway.assemblyai.com/v1/chat/completions"
CANDIDATES = ["claude-opus-5-5", "deepseek-v4.1-flash", "gemini-2.5-flash-lite"]  # swap for the live bench's Summarization picks

# The bench's own summarization task prompt (src/prompts/summarization.go)
PROMPT = """You are an expert call-transcript summarizer.

You will be given the raw transcript of a phone or video call. Write a summary that a busy person could read in place of the full call.

Rules:
- State only what is actually said in the transcript. Do not infer, assume, or add information that is not present.
- Capture the topics discussed, any decisions made, and any action items or commitments, in that rough order of importance.
- Omit small talk, filler words, and repeated statements.
- Write in plain prose, not bullet points, unless the call covers more than three distinct topics.
- Keep it short: a few sentences for a short call, a short paragraph for a long one. Do not pad the summary to sound more thorough than the call was.
- Output only the summary. No preamble, no "Here is a summary:", no closing remarks."""

for model in CANDIDATES:
    t0 = time.time()
    resp = requests.post(
        URL,
        headers={"authorization": API_KEY, "content-type": "application/json"},
        json={
            "model": model,
            "messages": [
                {"role": "system", "content": PROMPT},
                {"role": "user", "content": turns},
            ],
            "max_tokens": 600,
        },
    )
    resp.raise_for_status()
    elapsed = time.time() - t0
    summary = resp.json()["choices"][0]["message"]["content"]
    print(f"--- {model} ({elapsed:.2f}s) ---\n{summary}\n")

For a summary that renders progressively in a UI, add “stream”: true to the request body and read the server-sent events; the Gateway supports streaming on the same endpoint. For a longer walkthrough of transcribing and summarizing in Python, see how to summarize meetings with LLMs in Python.

Prompting for voice summaries

A few adjustments that consistently improve summaries of spoken transcripts, independent of the model:

• Pass speaker turns, not a wall of text. Utterance-formatted input lets the model attribute a decision to the person who made it, and it reduces the “the customer agreed” errors where it was the agent talking.
• Order it the way readers scan. The bench prompt asks for topics first, then decisions, then action items, in prose. If your product needs a task list with owners and due dates, request it as a separate labeled section (or a second call) so it isn’t buried inside a narrative paragraph.
• Scale the length to the call. A few sentences for a short call and a short paragraph for a long one, with bullets only when the call covers more than three distinct topics. A summary padded to look thorough loses on conciseness.
• Ask for grounded quotes. “For each point, include a short verbatim phrase from the transcript that supports it” makes faithfulness auditable and discourages inference.
• Chunk and merge at 60 minutes. If a model’s coverage drops at meeting length, summarize per 10- to 15-minute window with the same prompt, then run a final pass that merges the window summaries into one, with the instruction to keep every decision and owner. The merge pass is cheap and the per-window summaries are easier to spot-check.
• State the audience. “For the account manager who wasn’t on the call” produces a different, usually better, summary than no audience at all.

If you’d rather not maintain a prompt at all, Speech Understanding’s summarization returns a summary alongside the transcript in one request, with a summary_type setting that includes bullets and paragraph formats. It’s the right default when the summary format is stable; the Gateway path above is for when you want to control the model, the prompt, and the cost. Our earlier qualitative comparison of frontier models on summarization and the framework for choosing an LLM for voice are useful background; the bench adds the measured layer.

Test on your own calls

The bench shows which models to try first, not which to ship. Two ways to finish the job:

Playground compare mode: one transcript, one prompt, three models side by side. Fast enough to run on ten calls in an afternoon.

The repo for a measured run: the voice-standard-bench CLI accepts your own JSONL of transcripts. Set your API key and run go run . -models model-a,model-b from src/ to get faithfulness, coverage, and conciseness scores from the same judge setup on your data. Ten transcripts per length bucket is the bench’s own sample size; thirty gives you more confidence between close candidates.

Frequently Asked Questions (FAQs)

Which LLM is best for summarizing call transcripts?

It depends on call length. Voice Standard Bench scores every LLM Gateway model on summarization at 30 seconds, 2 minutes, 15 minutes, and 60 minutes, with Highest Performance, Lowest cost, and Fastest picks at each. As of October 28, 2026, the top scorers are gpt-5.6-terra at 30 seconds (100), claude-opus-5-5 at 2 and 15 minutes (99 and 97), and deepseek-v4.1-flash at 60 minutes (96). Most models score in the 90s on short calls; the spread widens at meeting length. Check the live Summarization tab at your length.

Does the best model change for long meetings?

Yes. At 60 minutes, coverage and faithfulness separate models more than at shorter lengths, and the speed advantage of small models narrows. The results page has a length picker for exactly this reason.

How is summarization quality measured on Voice Standard Bench?

Every model gets the same prompt: a plain-prose summary scaled to the call’s length, covering topics, then decisions, then action items. Two LLM judges score each summary for faithfulness, coverage, and conciseness on real speech-to-text output. The judges’ scores are tracked separately and combined; scores within 2 to 3 points should be read as a tie. Token-overlap metrics like ROUGE aren’t used because they reward phrasing over correctness.

What does “cost per hour of audio” mean?

It’s what one hour of audio costs to summarize with a given model at provider list price, accounting for input tokens, output tokens, and any reasoning tokens. It lets you budget in the unit a voice product runs on instead of converting from per-token prices by hand.

Can a small open-source model summarize as well as a frontier model?

On short transcripts, often close: at 2 minutes, nemotron-3-nano-30b-a3b scores 82 at about $0.0016 per hour of audio, against 99 for the top scorer. But the Lowest cost and Fastest cards are raw picks with no quality floor, so read the score next to each one. At meeting length, check the spread before deciding.

Should I use Speech Understanding summarization or pick my own Gateway model?

Use Speech Understanding when you want a summary returned with the transcript in one request and the format is stable. Use LLM Gateway when you want to choose the model, write the prompt, and control cost per hour, especially at scale or for long meetings.