> ## Documentation Index
> Fetch the complete documentation index at: https://assemblyai.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# Structured Outputs
## Overview
Structured outputs allow you to constrain the model's response to follow a specific JSON schema. This ensures the model returns data in a predictable format that can be reliably parsed and processed by your application.
To avoid JSON parse errors, add `post_processing_steps: [{"type": "json-repair"}]` to your request. The LLM Gateway will automatically repair common JSON errors before returning the response. See [Post-processing](#post-processing).
## Getting started
To use structured outputs, include the `response_format` parameter in your request with a `json_schema` type:
```python expandable theme={null}
import requests
headers = {
"authorization": "",
"content-type": "application/json"
}
response = requests.post(
"https://llm-gateway.assemblyai.com/v1/chat/completions",
headers=headers,
json={
"model": "gemini-2.5-flash-lite",
"messages": [
{
"role": "system",
"content": "You are a helpful math tutor. Guide the user through the solution step by step."
},
{
"role": "user",
"content": "how can I solve 8x + 7 = -23"
}
],
"response_format": {
"type": "json_schema",
"json_schema": {
"name": "math_reasoning",
"schema": {
"type": "object",
"properties": {
"steps": {
"type": "array",
"items": {
"type": "object",
"properties": {
"explanation": {"type": "string"},
"output": {"type": "string"}
},
"required": ["explanation", "output"],
"additionalProperties": False
}
},
"final_answer": {"type": "string"}
},
"required": ["steps", "final_answer"],
"additionalProperties": False
},
"strict": True
}
}
}
)
result = response.json()
print(result["choices"][0]["message"]["content"])
```
```javascript expandable theme={null}
const response = await fetch(
"https://llm-gateway.assemblyai.com/v1/chat/completions",
{
method: "POST",
headers: {
authorization: "",
"content-type": "application/json",
},
body: JSON.stringify({
model: "gemini-2.5-flash-lite",
messages: [
{
role: "system",
content:
"You are a helpful math tutor. Guide the user through the solution step by step.",
},
{
role: "user",
content: "how can I solve 8x + 7 = -23",
},
],
response_format: {
type: "json_schema",
json_schema: {
name: "math_reasoning",
schema: {
type: "object",
properties: {
steps: {
type: "array",
items: {
type: "object",
properties: {
explanation: { type: "string" },
output: { type: "string" },
},
required: ["explanation", "output"],
additionalProperties: false,
},
},
final_answer: { type: "string" },
},
required: ["steps", "final_answer"],
additionalProperties: false,
},
strict: true,
},
},
}),
}
);
const result = await response.json();
console.log(result.choices[0].message.content);
```
```bash expandable theme={null}
curl -X POST "https://llm-gateway.assemblyai.com/v1/chat/completions" \
-H "Content-Type: application/json" \
-H "Authorization: " \
-d '{
"model": "gemini-2.5-flash-lite",
"messages": [
{
"role": "system",
"content": "You are a helpful math tutor. Guide the user through the solution step by step."
},
{
"role": "user",
"content": "how can I solve 8x + 7 = -23"
}
],
"response_format": {
"type": "json_schema",
"json_schema": {
"name": "math_reasoning",
"schema": {
"type": "object",
"properties": {
"steps": {
"type": "array",
"items": {
"type": "object",
"properties": {
"explanation": { "type": "string" },
"output": { "type": "string" }
},
"required": ["explanation", "output"],
"additionalProperties": false
}
},
"final_answer": { "type": "string" }
},
"required": ["steps", "final_answer"],
"additionalProperties": false
},
"strict": true
}
}
}'
```
## Example response
When using structured outputs, the model's response will be a JSON string that conforms to your schema:
```json theme={null}
{
"request_id": "abc123",
"choices": [
{
"message": {
"role": "assistant",
"content": "{\"steps\":[{\"explanation\":\"Start with the equation 8x + 7 = -23\",\"output\":\"8x + 7 = -23\"},{\"explanation\":\"Subtract 7 from both sides to isolate the term with x\",\"output\":\"8x = -30\"},{\"explanation\":\"Divide both sides by 8 to solve for x\",\"output\":\"x = -30/8 = -15/4 = -3.75\"}],\"final_answer\":\"x = -3.75\"}"
},
"finish_reason": "stop"
}
],
"usage": {
"input_tokens": 85,
"output_tokens": 120,
"total_tokens": 205
}
}
```
You can parse the content as JSON in your application:
```python theme={null}
import json
content = result["choices"][0]["message"]["content"]
parsed = json.loads(content)
for step in parsed["steps"]:
print(f"{step['explanation']}: {step['output']}")
print(f"Final answer: {parsed['final_answer']}")
```
## Supported models
Structured outputs are supported by the following model families:
| Provider | Supported |
| ------------------------- | --------- |
| OpenAI (GPT-4.1, GPT-5.x) | Yes |
| Gemini | Yes |
| Claude (4.5+) | Yes |
| Alibaba Cloud Qwen | Yes |
| Moonshot AI Kimi | Yes |
| gpt-oss | No |
## Post-processing
Post-processing steps let you apply automatic fixes to model responses after generation. You can specify an ordered list of steps in the `post_processing_steps` parameter on any chat completions request. Steps run server-side on all LLM Gateway models in both US and EU regions.
Currently, JSON repair (`json-repair`) is the only supported step type.
### JSON repair
JSON repair corrects common JSON errors — such as trailing commas, unescaped characters, and missing quotes — that LLMs occasionally produce. This is especially useful when using structured outputs or [tool calling](/llm-gateway/tool-calling), where invalid JSON would otherwise require client-side retry logic.
#### Getting started
Add `post_processing_steps` to any chat completions request:
```python expandable theme={null}
import requests
import json
headers = {
"authorization": "",
"content-type": "application/json"
}
response = requests.post(
"https://llm-gateway.assemblyai.com/v1/chat/completions",
headers=headers,
json={
"model": "gemini-2.5-flash-lite",
"messages": [
{
"role": "user",
"content": "Extract the user name and age from: John is 30 years old. Return as JSON."
}
],
"post_processing_steps": [{"type": "json-repair"}]
}
)
result = response.json()
parsed = json.loads(result["choices"][0]["message"]["content"])
print(parsed)
```
```javascript expandable theme={null}
const response = await fetch(
"https://llm-gateway.assemblyai.com/v1/chat/completions",
{
method: "POST",
headers: {
authorization: "",
"content-type": "application/json",
},
body: JSON.stringify({
model: "gemini-2.5-flash-lite",
messages: [
{
role: "user",
content: "Extract the user name and age from: John is 30 years old. Return as JSON.",
},
],
post_processing_steps: [{ type: "json-repair" }],
}),
}
);
const result = await response.json();
const parsed = JSON.parse(result.choices[0].message.content);
console.log(parsed);
```
#### What JSON repair fixes
The JSON repair step corrects the most common JSON errors produced by LLMs:
| Error type | Example (broken) | After repair |
| ----------------------- | ---------------------- | ------------------------ |
| Trailing comma | `{"name": "John",}` | `{"name": "John"}` |
| Unescaped characters | `{"note": "say "hi""}` | `{"note": "say \"hi\""}` |
| Missing closing bracket | `{"name": "John"` | `{"name": "John"}` |
| Single-quoted strings | `{'name': 'John'}` | `{"name": "John"}` |
The step applies to both message content and tool call arguments in the response. `post_processing_steps` runs independently of `response_format`, so you can combine JSON repair with a `json_schema` for maximum reliability.
If the JSON cannot be repaired, the request returns an HTTP 500 error. The raw malformed response is never passed through.
## API reference
### Request parameters
The `response_format` parameter controls how the model formats its response:
| Key | Type | Required? | Description |
| ----------------------------- | ------ | --------- | ------------------------------------------------------------------------ |
| `response_format` | object | No | Specifies the format of the model's response. |
| `response_format.type` | string | Yes | The type of response format. Use `"json_schema"` for structured outputs. |
| `response_format.json_schema` | object | Yes | The JSON schema configuration object. |
### JSON schema object
| Key | Type | Required? | Description |
| -------------------- | ------- | --------- | -------------------------------------------------------------------------------------------- |
| `json_schema.name` | string | Yes | A name for the schema. Used for identification purposes. |
| `json_schema.schema` | object | Yes | A valid JSON Schema object that defines the structure of the expected response. |
| `json_schema.strict` | boolean | No | When `true`, the model will strictly adhere to the schema. Recommended for reliable parsing. |
### Schema definition
The `schema` object follows the [JSON Schema](https://json-schema.org/) specification. Common properties include:
| Property | Type | Description |
| ---------------------- | ------- | -------------------------------------------------------------------------- |
| `type` | string | The data type: `"object"`, `"array"`, `"string"`, `"number"`, `"boolean"`. |
| `properties` | object | For objects, defines the properties and their schemas. |
| `items` | object | For arrays, defines the schema for array items. |
| `required` | array | List of required property names. |
| `additionalProperties` | boolean | When `false`, prevents additional properties not defined in the schema. |
### Post-processing parameters
| Key | Type | Required? | Description |
| ------------------------------- | ------ | --------- | --------------------------------------------------------------------------------------------------- |
| `post_processing_steps` | array | No | An ordered list of post-processing steps to apply to the response. Omit to disable post-processing. |
| `post_processing_steps[i].type` | string | Yes | The step type. Currently `"json-repair"` is supported. |
Supported step types:
| Type | Applies to | Models | Regions |
| ------------- | --------------------------------------- | ---------------------- | --------- |
| `json-repair` | Message content and tool call arguments | All LLM Gateway models | US and EU |
## Best practices
When using structured outputs, keep these recommendations in mind:
Set `strict: true` to ensure the model's response strictly adheres to your schema. This is especially important when your application depends on specific fields being present.
Use `additionalProperties: false` at each level of your schema to prevent the model from adding unexpected fields to the response.
Keep your schemas focused and specific. Complex schemas with many nested levels may increase latency and token usage.
Include clear descriptions in your system or user messages to help the model understand what data to extract or generate for each field.
## Error handling
If the model cannot generate a valid response that matches your schema, you may receive an error or a response that doesn't fully conform to the schema. Always validate the parsed JSON against your expected structure:
```python theme={null}
import json
try:
content = result["choices"][0]["message"]["content"]
parsed = json.loads(content)
# Validate required fields exist
if "steps" not in parsed or "final_answer" not in parsed:
raise ValueError("Missing required fields in response")
except json.JSONDecodeError as e:
print(f"Failed to parse response as JSON: {e}")
except KeyError as e:
print(f"Unexpected response structure: {e}")
```