> ## Documentation Index
> Fetch the complete documentation index at: https://assemblyai.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# Choose a model
## Overview
LLM Gateway gives you access to dozens of models, and the best one depends on your task, how long your audio is, and whether you care most about quality, speed, or cost. The [models endpoint](/docs/llm-gateway/api-reference/list-available-models) answers this for you. Alongside each model's pricing and supported parameters, it returns a `performance` block with benchmark results for voice post-processing tasks.
The numbers come from [Voice Standard Bench](https://github.com/AssemblyAI/voice-standard-bench), AssemblyAI's open source benchmark. It runs every model behind LLM Gateway on real call transcripts and measures quality, latency, and cost.
Field names and values can change as the benchmark grows, so read it from the endpoint instead of copying numbers into your code.
## How the numbers work
### Tasks
Each model is tested on four voice post-processing tasks. Your model's score for each task is in `task_quality`.
| Task | Key in `task_quality` | What the model does | How it's scored |
| - | - | - | - |
| Summarization | `summarization` | Summarizes a call transcript. | An LLM judge scores faithfulness, coverage, and conciseness against the source transcript. |
| Information extraction | `information_extraction` | Pulls participants, action items, decisions, and entities from a transcript as JSON. | An LLM judge scores groundedness and completeness against the source transcript. |
| Text correction | `text_correction` | Cleans up disfluencies and errors in a transcript without changing its meaning. | An LLM judge scores faithfulness and improvement. |
| Dictation | `dictation` | Applies spoken commands such as "comma", "all caps", and "new paragraph". | Scored mechanically against an answer key, with no LLM judge. |
Quality scores run from 1 to 100, where higher is better. The overall `quality_score` is the equal-weight average of the tasks that have data.
### Audio length buckets
Every row in `performance.data` is for one audio length, in the `bucket` field: `30s`, `120s`, `900s`, or `3600s`. Pick the bucket closest to the transcripts you process. Scores generally drop as transcripts get longer, so a model that is excellent on 30-second calls can be a different choice for hour-long recordings.
### Fields
| Field | Meaning |
| - | - |
| `quality_score` | Combined quality, 1 to 100. |
| `task_quality` | Quality per task, 1 to 100. |
| `cost_per_hour_usd` | Estimated cost to process one hour of audio. Use this to compare cost across models, buckets, and tasks. `task_cost_per_hour_usd` gives the same estimate per task. |
| `total_cost_usd`, `cost_per_quality` | Cost of the benchmark samples behind the score. `cost_per_quality` is cost per call per quality point, in cents, where lower is more efficient. |
| `latency` | End-to-end request latency in seconds, with `p50`, `p95`, `mean`, `min`, `max`, and sample count `n`. |
| `ttft` | Time to first token in seconds from a separate streaming run, with the same statistics as `latency`. |
| `tokens_per_second` | Output speed. |
| `reasoning_effort` | The reasoning setting the row was measured with. See [Reasoning effort rows](#reasoning-effort-rows). |
### Reading the numbers carefully
* **A `0` means no data, not a bad score.** Dictation isn't measured on the `900s` and `3600s` buckets, so its `task_quality` is `0` there. A `ttft` of `0` means it wasn't measured. Skip zeros instead of ranking on them.
* **Small gaps are noise.** A one- or two-point difference between models doesn't tell you much. Prefer the cheaper or faster model when scores are close.
* **Cross-check outliers.** If a single task score is far above the model's `quality_score` or its other tasks, it may come from very few usable samples. Check `latency.n` and the other tasks before trusting it.
* **Look at `p95`, not just `p50`.** A model with a fast median can still have a slow tail, which matters for real-time use.
* **Costs use global pricing.** They don't include the regional surcharge shown on the [Models](/docs/llm-gateway/available-models#pricing-details) page. `cost_per_hour_usd` is an estimate based on average tokens per hour of audio, so fast or slow talkers shift your real cost.
* **The benchmark uses fixed prompts.** Your prompts, your audio, and your domain will differ. Treat the numbers as a way to build a shortlist, then test the shortlist on your own data.
### Reasoning effort rows
Most models have one row per bucket with `reasoning_effort` set to `default`. That means the benchmark didn't send the `reasoning_effort` parameter. A few models have extra rows measured with a specific value, such as `none`, `low`, or `high`. Each row is a separate configuration with its own quality, latency, and cost, so a lower setting can be much faster and cheaper at a small cost in quality.
To use a non-default row, send that value in the `reasoning_effort` parameter of your [chat completions](/docs/llm-gateway/chat-completions) request. Only models that list `reasoning_effort` in `supported_parameters` accept it.
## Pick a model
Work through these filters in order. They're the same ones the coding agent prompt below applies.
1. **Match the bucket.** Choose the `bucket` closest to your typical audio length. If your audio falls between buckets, use the longer one.
2. **Rank by the task you run.** Use `task_quality` for that task. Use `quality_score` only if you run several tasks with the same model.
3. **Filter to what you can use.** Drop models where `retirement_date` isn't `0`, models whose `available_regions` don't include your region (see [Cloud endpoints and data residency](/docs/llm-gateway/cloud-endpoints-and-data-residency)), and models whose `supported_parameters` lack what you need, such as `response_format` for [structured outputs](/docs/llm-gateway/structured-outputs) or `tools` for [tool calling](/docs/llm-gateway/tool-calling).
4. **Apply your constraint.** For lowest cost, sort by `cost_per_hour_usd`. For real-time responses, sort by `latency.p50` and check `latency.p95` (or `ttft` if you stream). For best quality, take the top score.
5. **Break ties on cost.** Among models within a couple of quality points of the top, choose the cheapest or fastest.
6. **Add a fallback.** Pick the runner-up as a [fallback](/docs/llm-gateway/fallback) so requests keep working if your primary model fails.
### Example from the data
This snapshot is from October 15, 2026. For each bucket it shows the top-scoring model and the cheapest model within 3 points of it, excluding models with a retirement date. Quality is the combined `quality_score`. The endpoint is the source of truth, so query it for current results.
| Audio length | Goal | Model | Quality | Cost per hour | p50 latency | Regions |
| - | - | - | - | - | - | - |
| `30s` | Top score | `gpt-6-luna` (`high`) | 96 | \$0.0054 | 2.5 s | US, global |
| `30s` | Cheapest within 3 points | `gpt-6-luna` (`none`) | 94 | \$0.0028 | 0.9 s | US, global |
| `120s` | Top score | `gemini-4-argon` | 94 | \$0.073 | 2.6 s | US, EU, global |
| `120s` | Cheapest within 3 points | `gemma-4-31b` (`none`) | 91 | \$0.0036 | 1.7 s | US |
| `900s` | Top score | `claude-opus-5` | 89 | \$0.32 | 14.9 s | US, global |
| `900s` | Cheapest within 3 points | `gpt-6-luna` | 87 | \$0.0048 | 15.5 s | US, global |
| `3600s` | Top score | `gpt-6-astra` | 88 | \$0.35 | 24.1 s | US |
| `3600s` | Cheapest within 3 points | `gpt-6-luna` | 85 | \$0.0039 | 19.0 s | US, global |
Models with no setting in parentheses were measured at `default` reasoning effort. Per-task winners differ from these overall winners, so rank on `task_quality` for your task before you decide.
## Query the models endpoint
The endpoint is public, so you don't need an API key. This example lists the five best models for summarization on 2-minute audio, skipping retiring models and models with no summarization data:
```bash theme={null}
curl -s https://llm-gateway.assemblyai.com/v1/models | jq -r '
[ .data[] | select(.retirement_date == 0) | . as $m
| .performance.data[]
| select(.bucket == "120s" and .task_quality.summarization > 0)
| { model: $m.id, reasoning_effort, summarization: .task_quality.summarization,
cost_per_hour_usd, p50_latency_s: .latency.p50 } ]
| sort_by(-.summarization) | .[:5][]
| [.model, .reasoning_effort, .summarization, .cost_per_hour_usd, .p50_latency_s] | @tsv'
```
Each model in the response has this shape. Fields not used for model selection are omitted here:
```json theme={null}
{
"id": "gpt-6-luna",
"available_regions": ["us", "global"],
"retirement_date": 0,
"supported_parameters": ["max_tokens", "tools", "tool_choice", "stream", "response_format", "reasoning_effort"],
"performance": {
"status": "beta",
"data": [
{
"model": "gpt-6-luna",
"reasoning_effort": "default",
"bucket": "30s",
"quality_score": 95,
"task_quality": { "dictation": 97, "information_extraction": 88, "summarization": 97, "text_correction": 96 },
"cost_per_hour_usd": 0.00427,
"latency": { "n": 100, "p50": 1.694, "p95": 5.055 },
"ttft": { "n": 50, "p50": 0.992, "p95": 3.565 }
}
]
}
}
```
To use the EU endpoint, replace `llm-gateway.assemblyai.com` with `llm-gateway.eu.assemblyai.com`.
## Choose a model with a coding agent
Describe what you're building in plain language, and your coding agent can read the endpoint, work out which tasks, audio length, and constraints apply, and write the integration code for the model it picks. Paste this prompt into Claude Code, Cursor, or your agent of choice, and replace the bracketed text with your description. For example: "a meeting notetaker that summarizes hour-long recordings and lists action items, with data kept in the EU."
```text theme={null}
Help me choose the best AssemblyAI LLM Gateway model for what I'm building, then wire it into my code.
What I'm building: [describe what you want to do with your audio or transcripts]
Work out my requirements from that description and from my codebase, instead of asking me for a list of settings. Infer:
- Which of the benchmark's tasks I need: summarization, information_extraction, text_correction, and/or dictation. A use case can need more than one, such as meeting notes (summarization and information_extraction) or cleaning up a transcript before display (text_correction).
- How long my audio is, and so which benchmark bucket to use: 30s, 120s, 900s, or 3600s.
- What I'm optimizing for: lowest cost, lowest latency, highest quality, or a balance. Real-time and interactive use points to latency. High-volume batch work points to cost. Clinical, legal, or customer-facing output points to quality.
- Which region I need. Mentions of EU, GDPR, or data residency point to eu.
- Which features I need, such as structured outputs (`response_format`), tool calling (`tools`), or streaming (`stream`).
State each of these as an assumption. Only ask me a question if the description leaves out something that would change the answer, and ask all such questions at once.
Then pick a model:
1. Fetch https://llm-gateway.assemblyai.com/v1/models. It is public and needs no API key.
2. Read https://www.assemblyai.com/docs/llm-gateway/choose-a-model.md to understand the `performance` fields before you rank anything.
3. Each model has `performance.data`, with one row per `bucket` and `reasoning_effort` value. Use the bucket closest to my audio length, rounding up to the longer bucket if it falls between two.
4. Rank on `task_quality[]` for each task I need. If I need several tasks with one model, use their average, or `quality_score` if all four tasks have data.
5. Treat a `task_quality` value of 0 as "not measured", never as a poor score. Dictation has no data for 900s or 3600s.
6. Drop any model that has a non-zero `retirement_date`, whose `available_regions` doesn't include my region, or whose `supported_parameters` is missing something I need.
7. Compare cost with `cost_per_hour_usd`. Compare latency with `latency.p50` and `latency.p95`, and `ttft` if I stream.
8. Treat scores within about 2 points as ties and break them with my priority.
9. Each `reasoning_effort` row is a separate configuration. If the best row is not "default", send that value as the `reasoning_effort` request parameter.
10. Show me a shortlist of 3 models as a table with the score for my task, cost per hour, p50 and p95 latency, bucket, and reasoning effort. Explain the tradeoffs in one or two sentences each, and recommend one.
After I pick a model:
- Write the code to call POST /v1/chat/completions on the LLM Gateway with that model, reading my API key from the ASSEMBLYAI_API_KEY environment variable. Use https://llm-gateway.eu.assemblyai.com if I need EU data residency.
- Set the runner-up from the shortlist as a fallback using the `fallbacks` parameter. See https://www.assemblyai.com/docs/llm-gateway/fallback.md
- Put the model ID and reasoning effort in configuration, not hardcoded in the logic.
Do not copy benchmark scores into the code or comments. They change. Remind me that the numbers come from fixed benchmark prompts, and suggest I test the shortlist on a few of my own transcripts before going to production.
```
To keep your agent on current information in an ongoing project, see [Build with AI agents](/docs/coding-agent-prompts) for how to pin AssemblyAI docs in your project instructions.
## Run Voice Standard Bench
The endpoint reflects one benchmark run on AssemblyAI's own transcripts and prompts. [Voice Standard Bench](https://github.com/AssemblyAI/voice-standard-bench) is open source (MIT), so you can see exactly how every number is produced and run it yourself.
**What it measures.** For every model behind LLM Gateway, it runs the four tasks above on transcripts grouped by audio length. It records latency, token usage, cost, and quality, and rolls them up into the fields you see in the endpoint.
**What the repository gives you:**
* **Published results.** Open `use_cases.html` in `results/canonical/` for a browsable report that groups the best models by use case and audio length, with no setup. `summary.csv` and `summary.json` hold every row.
* **The prompts.** The task prompts, judge prompts, and scoring rubrics live in `src/prompts`. Read them to see what a score is measuring.
* **The harness.** A Go CLI that benchmarks models through LLM Gateway. Use it to benchmark a specific shortlist of models, try a different `reasoning_effort`, or replay a run with a different judge without calling the models again.
**When to run it yourself.** Run it when the published numbers aren't enough to decide, for example if your transcripts look very different from call audio, or you want to compare models on your own data. Otherwise the endpoint and the published results are the faster route.
The benchmark reads its transcripts from a `normalized/` directory that isn't included in the repository, and running it makes billable LLM Gateway calls. Read the [README](https://github.com/AssemblyAI/voice-standard-bench#readme) before you start. It covers the dataset layout, the flags, and how scores are calculated.
## Related
* [Models](/docs/llm-gateway/available-models): every model with its ID, parameters, and pricing.
* [Model fallbacks](/docs/llm-gateway/fallback): automatically retry on a backup model.
* [Cloud endpoints and data residency](/docs/llm-gateway/cloud-endpoints-and-data-residency): which models are available in which regions.
* [Improve latency](/docs/llm-gateway/improve-latency): reduce request overhead with keep-alive connections.