Best LLMs for Extracting Structured Data from Transcripts (JSON Output, Speed, and Cost)

Search for “best LLM for structured data extraction” and every result is about PDFs, invoices, and web pages. None of them cover the input most voice products deal with: a transcript of someone talking. That’s a different problem. The caller said “my order number is, uh, four seven, no, four eight two one,” the agent repeated it back wrong, and your schema has one field for it.

This post covers how to pick a model for pulling participants, action items, decisions, names, and dates out of spoken transcripts. It leans on Voice Standard Bench, which scores every model on LLM Gateway on exactly this task, at four transcript lengths. The picks below are as of October 28, 2026. The live page is the source of truth, and it changes as models change.

Why extraction from transcripts is harder than from documents

A PDF says what it says. A transcript says what the speech-to-text model heard, which is close to what the speaker said, which is sometimes not what the speaker meant. Four things make spoken extraction its own task:

Misheard and spelled-out numbers. “Fifteen” and “fifty” are a phoneme apart. Callers read digits in groups, restart halfway, and say “double four.” A model has to reconstruct the intended value from a messy sequence, and it has to know when it can’t.

Spelled names and hedging. “It’s Siobhan, S-I-O-B-H-A-N.” The name and the spelling arrive as separate tokens, often with the transcript’s own guess in between. “I think it was the 14th, or maybe the 15th” has no single correct date.

Speaker attribution. On a two-party call, the callback number might be the agent reading back a wrong value, not the caller stating the right one. Extraction without speaker labels pulls the wrong turn.

Cross-talk and disfluency. Overlapping speech, fillers, and false starts sit between the model and the fact. A model trained to produce fluent text can smooth over a gap by inventing something plausible, which is the single worst outcome in extraction.

That last point is why the bench’s extraction rubric leads with groundedness, and why “which model produces the nicest JSON” is the wrong question. The right one is “which model returns only what was actually said, in the shape I asked for, fast enough for my pipeline.”

How Voice Standard Bench scores information extraction

Voice Standard Bench runs every Gateway model (50+ at the time of writing) on real speech-to-text output across four tasks: summarization, information extraction, text correction, and dictation cleanup. Information extraction asks the model to fill one fixed JSON shape from a call transcript: the participants, the action items (each with an owner, a task, and a due date), the decisions reached, and the named entities mentioned (people, organizations, products, and dates).

The prompt is strict about that shape. Every key must be present. A field with nothing to report gets an empty array, and an owner or due date that was never said gets “unspecified”, not null and not a guess. The model must output only the JSON object, with no preamble and no markdown fence.

Scoring then happens in two steps:

• Schema check first. Before any judge sees the output, the bench checks it mechanically: is it a JSON object, and are all four required keys present? An output that fails is never judged. It gets the worst possible rubric score, so schema validity is folded straight into the task score. The bench also counts these schema failures separately, so a model that reliably breaks the shape shows up as its own failure mode.
• Two LLM judges on content. Valid outputs are graded by two judges (Claude Opus 5.5 and GPT-5.6 Sol, tracked separately and then combined) on groundedness (is every extracted value supported by the transcript? Invented fields fail here) and completeness (did the model capture the action items, decisions, and entities that were there to find?).

The task runs at four lengths: 30 seconds (a voicemail), 2 minutes (a short call), 15 minutes (a support call), and 60 minutes (a meeting), with 10 real transcripts per bucket, each within 15% of the target length. Each model also gets a median response time and a cost per hour of audio at provider list price. Requests that error out (timeouts, provider errors) are recorded but left out of the score, rather than counted as zero. The methodology post explains the rubric and the judge calibration in full: the two judges differ by a few points on average, and self-preference bias was measured and found small.

Four questions before you pick a model

1. How strict is your schema? If downstream code parses the output, you need valid JSON every time. The bench score already punishes a broken shape, so a high extraction score means the model held the shape on the bench’s transcripts. On your own pipeline, prefer models that support structured outputs through response_format, and still validate every response.

2. How bad is a hallucinated field? A wrong callback number in a CRM is worse than an empty one. Weight groundedness over completeness, and give the model a legitimate way to say “not stated”: the bench’s convention is an empty array, or “unspecified” for an owner or due date, never an omitted key and never a guess.

3. What’s your latency budget? Extracting intent mid-call to route a voice agent needs a response in well under a second, which points at the Fastest card. Check its score first. Nightly extraction over yesterday’s recordings can take the top performer regardless of speed.

4. How long are your transcripts? This changes the answer more than most people expect. Short inputs favor small, fast, cheap models. At meeting length, larger models pull ahead on completeness. Set the length picker before you read any ranking.

The choose a model docs show how to filter the Gateway catalog by task and length and build a shortlist of two or three candidates.

Top models for transcript extraction by length

On the live results page, open the Information extraction tab and set the length. Each view shows three cards: Highest Performance (the top score; ties go to the lower cost), Lowest cost (the lowest cost per hour of audio), and Fastest (the lowest median response time at that length). Lowest cost and Fastest are raw picks with no quality floor, so check the score before you ship one. Here is each card with its score, as of October 28, 2026:

*nemotron-lightning-3p5-30b-a3b tops the 2-minute view with a score of 100, but read it as an outlier. The bench logged 15 errored requests for this model across its 2-minute runs, and errored requests are left out of the score rather than counted against it. Its median response time at this length is 18.1 seconds, and at 30 seconds the same model scores 26 on extraction. The next-highest 2-minute scores are gpt-5.6-luna and gemini-4-argon, at 91 each.

Scores within 2 to 3 points of each other differ only marginally. If you want picks with a quality floor, the take-home tables in the Voice Standard Bench launch post list the cheapest and fastest models that land within 8 points of the top scorer for each task. The live cards above don’t apply that rule.

How to read the shift across lengths: on voicemails and short calls, the facts are few and close together, so a small model finds them and returns in a fraction of the time. At 30 seconds, 43 of the 52 models on the tab score 80 or higher. On a 15-minute support call, the order number appears once, gets corrected ten minutes later, and the final value is in the agent’s read-back. Completeness starts to separate models there, and the top score drops to 85. At 60 minutes, context handling dominates: some models drop facts from the middle of a long transcript, and only 9 models score 80 or higher. The Fastest pick tells the same story: qwen3.5-4b-32k-fast scores 80 at 30 seconds and 33 at 15 minutes. The launch post’s headline finding still holds (“most voice tasks don’t need frontier models, or frontier-model prices”), but the exception is most visible on this task at the longest buckets.

Getting valid JSON out of the Gateway

LLM Gateway is OpenAI-compatible, so structured outputs work the way you’d expect: pass response_format with a JSON schema, set strict to true, and the model’s output is constrained to the schema. Structured outputs are supported on OpenAI, Gemini, Claude (4.5 and later), Qwen, and Kimi models on the Gateway, and not on gpt-oss; check the structured outputs docs for the current list. Same API key as Speech-to-Text, provider list price with 0% markup, and switching models means changing one string.

The sample below uses the bench’s own extraction prompt and the same four-key shape, so what you test matches what the bench scored. It loops over three candidate model IDs so you can compare output and response time in one run, and it repeats the bench’s schema check on every response. The bench itself sends the shape in the prompt only; the sample also passes it as a strict response_format, which constrains models that support structured outputs. Swap the IDs for whatever the Information extraction tab shows at your length.

import os, json, time, requests

API_KEY = os.environ["ASSEMBLYAI_API_KEY"]
URL = "https://llm-gateway.assemblyai.com/v1/chat/completions"

# Swap for the models on the live bench's Information extraction tab at your length
CANDIDATES = ["gpt-5.6-sol", "claude-haiku-5-5", "qwen3.5-4b-32k-fast"]

STRINGS = {"type": "array", "items": {"type": "string"}}

# The same four-key shape Voice Standard Bench asks every model for
SCHEMA = {
    "type": "object",
    "properties": {
        "participants": STRINGS,
        "action_items": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "owner": {"type": "string"},
                    "task": {"type": "string"},
                    "due": {"type": "string"},
                },
                "required": ["owner", "task", "due"],
                "additionalProperties": False,
            },
        },
        "decisions": STRINGS,
        "entities": {
            "type": "object",
            "properties": {
                "people": STRINGS,
                "organizations": STRINGS,
                "products": STRINGS,
                "dates": STRINGS,
            },
            "required": ["people", "organizations", "products", "dates"],
            "additionalProperties": False,
        },
    },
    "required": ["participants", "action_items", "decisions", "entities"],
    "additionalProperties": False,
}

# The bench's own task prompt (src/prompts/extraction.go)
SYSTEM = """You are an expert at extracting structured information from call transcripts.

You will be given the raw transcript of a phone or video call. Extract information into EXACTLY this JSON shape:

{
  "participants": ["<name or role of each speaker who can be identified, e.g. "Alice" or "the caller">"],
  "action_items": [{"owner": "<who is responsible, or "unspecified">", "task": "<what they agreed to do>", "due": "<when, or "unspecified">"}],
  "decisions": ["<a concrete decision or agreement reached during the call>"],
  "entities": {"people": ["<named individuals mentioned who are not speakers>"], "organizations": ["<company or team names mentioned>"], "products": ["<product, tool, or project names mentioned>"], "dates": ["<specific dates or deadlines mentioned>"]}
}

Rules:
- Include ONLY things actually stated in the transcript. Do not infer a due date, owner, or decision that was not said.
- If a field has nothing to report, use an empty array (or "unspecified" for owner/due) — do not omit the key.
- Do not extract the same fact into two different fields.
- Output ONLY the JSON object. No preamble, no markdown code fence, no commentary before or after it."""

REQUIRED_KEYS = ["participants", "action_items", "decisions", "entities"]


def validate(content):
    """Mirror the bench's mechanical check: valid JSON with every required key present."""
    data = json.loads(content)
    missing = [key for key in REQUIRED_KEYS if key not in data]
    if missing:
        raise ValueError(f"missing required key(s): {', '.join(missing)}")
    return data


transcript = open("call.txt").read()

for model in CANDIDATES:
    t0 = time.time()
    resp = requests.post(
        URL,
        headers={"authorization": API_KEY, "content-type": "application/json"},
        json={
            "model": model,
            "messages": [
                {"role": "system", "content": SYSTEM},
                {"role": "user", "content": transcript},
            ],
            "response_format": {
                "type": "json_schema",
                "json_schema": {"name": "call_extraction", "schema": SCHEMA, "strict": True},
            },
            "max_tokens": 1000,
        },
    )
    resp.raise_for_status()
    elapsed = time.time() - t0
    content = resp.json()["choices"][0]["message"]["content"]
    try:
        data = validate(content)
    except ValueError as err:  # json.JSONDecodeError is a ValueError too
        data = {"_error": str(err), "_raw": content}
    print(f"--- {model} ({elapsed:.2f}s) ---")
    print(json.dumps(data, indent=2))

Three habits that keep extraction honest in production:

• Validate and retry. Even with strict schemas, parse the output and check that every key is there. On a failure, retry once with the error message appended to the prompt, then fall back to empty fields rather than blocking the pipeline.
• Empty, not missing. A schema where every list can be empty, and every owner or due date can be “unspecified”, gives the model a legitimate way to say “not stated.” A schema that demands a value invites a guess.
• Tool calling as an alternative. The Gateway also supports tools and tool_choice. Defining your schema as a function and forcing the call gets you the same structured result, and it fits naturally if the extraction feeds an agent that then acts on the data.

If you’re starting from audio, transcribe first with the Speech-to-Text API (set speech_models to universal-3-5-pro and enable speaker labels), then pass the utterance-formatted transcript in. The apply LLMs to audio files guide shows the handoff, and the Gemini 4 Argon post has a second worked Gateway example.

Short-call vs long-call extraction patterns

Under 15 minutes: send the whole transcript in one request with speaker labels. Put the schema in response_format, keep the system prompt short, and consider the Fastest or Lowest cost cards if their score at your length holds up and groundedness on your data agrees.

15 to 60 minutes: a single pass works on most models, but watch for facts in the middle of the transcript going missing. Two patterns help. First, chunk by speaker turn or by five-minute window, extract per chunk with the same schema, then merge with a rule for conflicts (latest confirmed value wins). Second, pre-summarize: ask the model for a tight, grounded summary with every number and name preserved, then extract from that. The second pass is cheaper and the summary is easier to audit.

When you don’t need to pick a model at all: Speech Understanding runs extraction-style tasks directly on the transcript without an LLM selection step. If your fields are stable and you’d rather not maintain a prompt, start with the Speech Understanding docs. If your schema changes often or you want control over the model and cost, the Gateway path above is the better fit. An earlier, qualitative look at extraction prompts across frontier models is in our Gemini 3 Pro vs GPT-5 vs Claude comparison; the bench supersedes it with measured results.

Test on your transcripts

The bench tells you which models to try first, not which to ship. Two ways to close the gap:

Playground compare mode for a quick read: one transcript, one prompt, three models side by side.

The repo for a real evaluation. The voice-standard-bench CLI runs on your own JSONL of transcripts. Set your API key, run go run . -models model-a,model-b from src/, and you get the same schema check plus groundedness and completeness scores from the same judge setup on your data. Thirty transcripts per length bucket with a hand-written answer key is enough to rank your shortlist with confidence.

Frequently Asked Questions (FAQs)

Which LLM is best for extracting data from call transcripts?

It depends on transcript length. Voice Standard Bench scores every LLM Gateway model on information extraction at 30 seconds, 2 minutes, 15 minutes, and 60 minutes, with Highest Performance, Lowest cost, and Fastest picks for each. As of October 28, 2026, the top extraction scores are gpt-5.6-sol at 30 seconds (94), gpt-6-luna at 15 minutes (85), and gpt-6.1-sol at 60 minutes (85). Check the live Information extraction tab, then test your top two or three on your own calls.

Does LLM Gateway support JSON schema and structured output?

Yes. Pass response_format with type: “json_schema”, your schema, and strict: true, the same way you would with the OpenAI API. Structured outputs are supported on OpenAI, Gemini, Claude 4.5 and later, Qwen, and Kimi models on the Gateway, but not on gpt-oss. Tool calling with tools and tool_choice is an alternative.

How does transcript length change the best model?

On short inputs, many models score well, and small models win on speed and cost. At 15 and 60 minutes, larger models pull ahead on completeness because facts are spread across more context, and fewer models stay above 80. Always set the length before reading the ranking.

How do I stop a model from inventing fields it didn't hear?

Give it a legitimate way to say “not stated”: an empty array for a list with nothing in it, and “unspecified” for an owner or due date that wasn’t said. Tell it to include only what the transcript states, and how to handle spelled-out names and corrected numbers. Then validate every output and prefer models with a high extraction score on the bench, since that score already reflects groundedness and schema validity.

Is a small open model good enough for extraction?

Often, for short calls and voicemails. At 30 seconds, qwen3.5-4b-32k-fast scores 80 and answers in 0.28 seconds, against a top score of 94. But Lowest cost and Fastest are raw picks with no quality floor, and small models fall off on long transcripts: the same model scores 33 at 15 minutes. Check the score at your length before committing.

How is information extraction scored on Voice Standard Bench?

Each output is first checked mechanically: it must be a JSON object with all four required keys. An output that fails gets the worst possible score, so schema validity counts toward the result. Valid outputs are then scored by two LLM judges for groundedness (every value is supported by the transcript) and completeness (nothing load-bearing was missed). Cost per hour of audio and median response time round out the three figures on the results page.