New AssemblyAI-TTS-1: Text-to-speech for voice agents. Learn more
LLM Gateway

Run open and frontier LLMs on your voice data

From raw utterance to structured answer—fast open models for the cleanup pass, frontier models for the reasoning pass, one API key for both.

  • 60 open and frontier models from 7 providers, all OpenAI-compatible
  • Your LLM calls run where your Speech-to-Text does—no second vendor, no extra hop
  • 0% markup on global routing, automatic fallbacks, EU data residency, and zero data retention

Open and frontier, one endpoint

Fast open models for the cleanup pass, frontier models for the reasoning pass. 60 models from 7 providers behind one API key—switch by changing one string.

Voice-native

Your LLM calls run where your Speech-to-Text does—one less network hop on every turn, no second vendor, and one key for the whole pipeline.

0% markup

Pay provider rates, not gateway rates, on global routing. Competing gateways add 5% or more to every call. No hidden fees, no minimum commitment.

Metaview
Ashby
Cluely
Genio
Siro

36%

improvement in close rate

See case study
LiveKit
EliseAI
Commure
Dovetail
Fireflies

“The new Universal-3.5 Pro speech model from AssemblyAI is best so far in terms of accuracy, latency, and language switching.”

Retell
CallRail
Apollo.io
ClickUp
Calabrio

80%

increase in customer satisfaction

HeyGen
Granola
Siro
JotPsych
Granola

“Assembly has saved us countless hours managing models, and provided exceptional accuracy.”

Metaview
Ashby
Cluely
Genio
Siro

36%

improvement in close rate

See case study
LiveKit
EliseAI
Commure
Dovetail
Fireflies

“The new Universal-3.5 Pro speech model from AssemblyAI is best so far in terms of accuracy, latency, and language switching.”

Retell
CallRail
Apollo.io
ClickUp
Calabrio

80%

increase in customer satisfaction

HeyGen
Granola
Siro
JotPsych
Granola

“Assembly has saved us countless hours managing models, and provided exceptional accuracy.”

Live demo

Pick a model. Take action on your audio.

Prompt

What is runner's knee?

claude-opus-4-7 412 ms · 47 tokens

Based on the transcript, runner's knee is a condition characterized by pain behind or around the kneecap. It is caused by overuse, muscle imbalance and inadequate stretching. Symptoms include pain under or around the kneecap and pain when walking.

Easiest, most reliable way to call multiple LLMs

Ship faster, spend less on tokens, and stop losing users to provider outages.

Models worth using

Frontier models from OpenAI, Anthropic, and Google, plus open models we host ourselves. New ones added the day they launch.

Automatic fallbacks

Configure backup models per request. When a provider errors, stalls, or rate-limits, your call still goes through—no code changes.

OpenAI-compatible

Drop into any OpenAI SDK or HTTP client. Change a base URL and a model string, and the rest of your code keeps working.

Function calls & typed JSON

Tool calls and schema-enforced JSON out of the box, so model responses drop straight into your app.

Prompt caching

Cached prompts cost less and respond faster on every turn of a conversation.

EU data residency

EU-resident infrastructure, on by default for traffic originating in the EU, for GDPR-sensitive workloads.

Zero data retention

Opt into zero data retention per request or project-wide, so prompts and responses are never stored.

BAA available

A Business Associate Agreement is available for customers processing PHI, backed by a SOC 2 Type 2 report.

Quickstart

Your first Gateway call, minutes after you sign up

Create a free account, point your OpenAI SDK at the Gateway, and start calling any open or frontier model. Already using AssemblyAI Speech-to-Text? It's the same API key.

Start building free

No credit card required

Frequently asked questions