LLM Gateway Overview

Overview

AssemblyAI’s LLM Gateway is a unified interface that allows you to connect with multiple LLM providers including Claude, GPT, Gemini, and more. You can use the LLM Gateway to build sophisticated AI applications through a single API.

The LLM Gateway provides access to 25+ models across major AI providers with support for:

• Basic Chat Completions - Simple request/response interactions
• Streamed Responses - Stream output as it’s generated (OpenAI models)
• Multi-turn Conversations - Maintain context across multiple exchanges
• Structured Outputs - Constrain responses to a specific JSON schema
• Tool/Function Calling - Enable models to execute custom functions
• Agentic Workflows - Multi-step reasoning with automatic tool chaining
• Unified Interface - One API for Claude, GPT, Gemini, Qwen, Kimi, and more
• Post-processing - Automatically repair malformed JSON responses with built-in JSON repair

Available models

By quality (LMArena Score)

By latency (per 10,000 tokens)

By provider

Anthropic Claude

OpenAI GPT

Google Gemini

Alibaba Cloud Qwen

Moonshot AI Kimi

Select a model

You can specify which model to use in your request by setting the model parameter. Here are examples showing how to use Claude 4.5 Sonnet:

1
import requests
2
3
headers = {
4
  "authorization": "<YOUR_API_KEY>"
5
}
6
7
response = requests.post(
8
    "https://llm-gateway.assemblyai.com/v1/chat/completions",
9
    headers = headers,
10
    json = {
11
        "model": "claude-sonnet-4-6",
12
        "messages": [
13
            {"role": "user", "content": "What is the capital of France?"}
14
        ],
15
        "max_tokens": 1000
16
    }
17
)
18
19
result = response.json()
20
print(result["choices"][0]["message"]["content"])

Simply change the model parameter to use any of the available models listed in the Available models section above.

Next steps

• Basic Chat Completions - Learn how to send simple messages and receive responses
• Multi-turn Conversations - Maintain context across multiple exchanges
• Structured Outputs - Constrain model responses to follow a specific JSON schema
• Tool Calling - Enable models to execute custom functions
• Agentic Workflows - Build multi-step reasoning applications
• Post-processing - Automatically repair malformed JSON in model responses