> ## Documentation Index
> Fetch the complete documentation index at: https://assemblyai.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Agentic Workflows

## Overview

Agentic workflows enable the model to make multiple sequential tool calls for complex multi-step reasoning. The model autonomously decides which tools to use, chains them together, and iterates until it has sufficient information to answer the question.

<Tip>
  Because every iteration depends on the model emitting valid JSON in `tool_calls[i].function.arguments`, a single malformed argument can break the entire chain. Add `post_processing_steps: [{"type": "json-repair"}]` to your request to automatically repair common JSON errors before they reach your loop. See [Post-processing](/llm-gateway/structured-outputs#post-processing).
</Tip>

## Getting started

Enable the model to make multiple sequential tool calls:

```python expandable theme={null}
import requests

headers = {
  "authorization": "<YOUR_API_KEY>"
}

conversation_history = [
    {"role": "user", "content": "Where does Sarah work and what city is that in?"}
]

max_iterations = 10
iteration = 0

while iteration < max_iterations:
    iteration += 1
    response = chat(conversation_history, model)
    choice = response["choices"][0]

    if choice.get("message", {}).get("tool_calls"):
        # Execute tools and add results to history
        handle_tool_calls(choice["message"]["tool_calls"], conversation_history)
        # Continue loop - model can make more tool calls
        continue
    else:
        # Model has final answer
        print(choice["message"]["content"])
        break
```

## Example interaction

```
You: Where does Andrew live?
🔧 Calling: search_employees(first_name="Andrew")
✅ Result: Found 1 result
🔧 Calling: search_tavily(query="33186 zip code location")
✅ Result: The zip code 33186 is located in Miami, Florida...
Assistant: Andrew lives in Miami, Florida (zip code 33186).
```

## How agentic behavior works

The model autonomously:

* Decides which tools to use based on the question
* Chains multiple tools together (e.g., get employee info → lookup location details)
* Iterates until it has sufficient information to answer
* Handles complex multi-step reasoning automatically

This is achieved by running a loop that continues making requests as long as the model is calling tools.

## API reference

### Request

The LLM Gateway accepts POST requests to `https://llm-gateway.assemblyai.com/v1/chat/completions` with the following parameters:

```bash expandable theme={null}
curl -X POST \
  "https://llm-gateway.assemblyai.com/v1/chat/completions" \
  -H "Authorization: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-sonnet-4-6",
    "messages": [
      {
        "role": "user",
        "content": "Where does Sarah work and what city is that in?"
      }
    ],
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "search_employees",
          "description": "Search employee database",
          "parameters": {
            "type": "object",
            "properties": {
              "first_name": {
                "type": "string"
              }
            }
          }
        }
      },
      {
        "type": "function",
        "function": {
          "name": "search_web",
          "description": "Search the web for information",
          "parameters": {
            "type": "object",
            "properties": {
              "query": {
                "type": "string"
              }
            }
          }
        }
      }
    ],
    "max_tokens": 1000
  }'
```

#### Request parameters

| Key           | Type             | Required? | Description                                                                                                          |
| ------------- | ---------------- | --------- | -------------------------------------------------------------------------------------------------------------------- |
| `model`       | string           | Yes       | The model to use for completion. See [Available models](/llm-gateway#available-models) section for supported values. |
| `messages`    | array            | Yes       | An array of message objects representing the conversation history.                                                   |
| `tools`       | array            | Yes       | An array of tool definitions that the model can call.                                                                |
| `tool_choice` | string or object | No        | Controls which tools the model can call. Options: `"none"`, `"auto"`, or an object specifying a specific function.   |
| `max_tokens`  | number           | No        | The maximum number of tokens to generate. Default: 1000. Range: \[1, context\_length).                               |
| `temperature` | number           | No        | Controls randomness in the output. Higher values make output more random. Range: \[0, 2].                            |

#### Message object

| Key            | Type            | Required? | Description                                                                                                    |
| -------------- | --------------- | --------- | -------------------------------------------------------------------------------------------------------------- |
| `role`         | string          | Yes       | The role of the message sender. Valid values: `"user"`, `"assistant"`, `"system"`, or `"tool"`.                |
| `content`      | string or array | Yes       | The message content. Can be a string or an array of content parts for the `"user"` role.                       |
| `name`         | string          | No        | An optional name for the message sender. For non-OpenAI models, this will be prepended as `{name}: {content}`. |
| `tool_call_id` | string          | No\*      | Required when `role` is `"tool"`. The ID of the tool call this message is responding to.                       |

#### Tool object

| Key                    | Type   | Required? | Description                                                 |
| ---------------------- | ------ | --------- | ----------------------------------------------------------- |
| `type`                 | string | Yes       | The type of tool. Currently only `"function"` is supported. |
| `function`             | object | Yes       | The function definition.                                    |
| `function.name`        | string | Yes       | The name of the function.                                   |
| `function.description` | string | No        | A description of what the function does.                    |
| `function.parameters`  | object | Yes       | A JSON Schema object describing the function parameters.    |

#### Tool choice object

The `tool_choice` parameter can be:

* `"none"` - The model will not call any tools
* `"auto"` - The model can choose whether to call tools
* An object with `type: "function"` and a `function.name` to force calling a specific function

### Response

The API returns a JSON response. In agentic workflows, the model may make multiple tool calls before providing a final answer.

#### First response (tool call)

```json expandable theme={null}
{
  "request_id": "abc123",
  "choices": [
    {
      "index": 0,
      "finish_reason": "tool_use",
      "message": {
        "role": "assistant",
        "content": "",
        "tool_calls": [
          {
            "id": "call_123",
            "type": "function",
            "function": {
              "name": "search_employees",
              "arguments": "{\"first_name\": \"Sarah\"}"
            }
          }
        ]
      }
    }
  ],
  "request": { "model": "claude-sonnet-4-6", "max_tokens": 1000 },
  "usage": { "input_tokens": 150, "output_tokens": 30, "total_tokens": 180 }
}
```

#### Subsequent response (another tool call)

After adding the function result to conversation history:

```json expandable theme={null}
{
  "request_id": "def456",
  "choices": [
    {
      "index": 0,
      "finish_reason": "tool_use",
      "message": {
        "role": "assistant",
        "content": "",
        "tool_calls": [
          {
            "id": "call_456",
            "type": "function",
            "function": {
              "name": "search_web",
              "arguments": "{\"query\": \"Acme Corp headquarters location\"}"
            }
          }
        ]
      }
    }
  ],
  "request": { "model": "claude-sonnet-4-6", "max_tokens": 1000 },
  "usage": { "input_tokens": 220, "output_tokens": 35, "total_tokens": 255 }
}
```

#### Final response (answer)

After all tool calls are complete:

```json expandable theme={null}
{
  "request_id": "ghi789",
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "Sarah works at Acme Corp, which is headquartered in San Francisco, California."
      },
      "finish_reason": "stop"
    }
  ],
  "request": {
    "model": "claude-sonnet-4-6",
    "max_tokens": 1000
  },
  "usage": {
    "input_tokens": 280,
    "output_tokens": 20,
    "total_tokens": 300
  }
}
```

#### Response fields

| Key                             | Type   | Description                                                                                                                                                                                                                                              |
| ------------------------------- | ------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `request_id`                    | string | A unique identifier for the request.                                                                                                                                                                                                                     |
| `choices`                       | array  | An array of completion choices. Typically contains one choice.                                                                                                                                                                                           |
| `choices[i].message`            | object | The message object containing the model's response.                                                                                                                                                                                                      |
| `choices[i].message.role`       | string | The role of the message, typically `"assistant"`.                                                                                                                                                                                                        |
| `choices[i].message.content`    | string | The text content of the model's response. May be null when calling tools.                                                                                                                                                                                |
| `choices[i].finish_reason`      | string | The reason the model stopped generating. Common values: `"stop"`, `"length"`, `"tool_calls"` (OpenAI models), `"tool_use"`, `"end_turn"` (Claude models). LLM Gateway passes the provider-native value through; client code should accept either family. |
| `choices[i].message.tool_calls` | array  | Present on assistant message choices when the model calls a tool. Each tool call appears in its own choice with the calling content nested under `message`.                                                                                              |
| `request`                       | object | Echo of the request parameters (excluding `messages`).                                                                                                                                                                                                   |
| `usage`                         | object | Token usage statistics for the request.                                                                                                                                                                                                                  |
| `usage.input_tokens`            | number | Number of tokens in the prompt.                                                                                                                                                                                                                          |
| `usage.output_tokens`           | number | Number of tokens in the completion.                                                                                                                                                                                                                      |
| `usage.total_tokens`            | number | Total tokens used (prompt + completion).                                                                                                                                                                                                                 |

#### Tool call object

When the model wants to call a tool, the response includes a `tool_calls` array:

| Key                                | Type   | Description                                      |
| ---------------------------------- | ------ | ------------------------------------------------ |
| `tool_calls[i].id`                 | string | A unique identifier for the tool call.           |
| `tool_calls[i].type`               | string | The type of tool call, always `"function"`.      |
| `tool_calls[i].function`           | object | The function call details.                       |
| `tool_calls[i].function.name`      | string | The name of the function to call.                |
| `tool_calls[i].function.arguments` | string | A JSON string containing the function arguments. |

### Error response

If an error occurs, the API returns an error response:

```json theme={null}
{
  "code": 400,
  "message": "invalid request body",
  "request_id": "2a9adf03-c73e-4333-a42d-54b515e6afbd",
  "metadata": {
    "errors": [
      "one of messages or prompt required"
    ]
  }
}
```

| Key               | Type   | Description                                                              |
| ----------------- | ------ | ------------------------------------------------------------------------ |
| `code`            | number | HTTP status code for the error.                                          |
| `message`         | string | A human-readable description of the error.                               |
| `request_id`      | string | Unique identifier for the request. Include this when contacting support. |
| `metadata`        | object | Optional. Present on 400 responses with per-field validation details.    |
| `metadata.errors` | array  | List of specific validation failure messages.                            |

#### Common error codes

| Code | Description                               |
| ---- | ----------------------------------------- |
| 400  | Bad Request - Invalid request parameters  |
| 401  | Unauthorized - Invalid or missing API key |
| 403  | Forbidden - Insufficient permissions      |
| 404  | Not Found - Invalid endpoint or model     |
| 429  | Too Many Requests - Rate limit exceeded   |
| 500  | Internal Server Error - Server-side error |