Agentic Workflows

Overview

Agentic workflows enable the model to make multiple sequential tool calls for complex multi-step reasoning. The model autonomously decides which tools to use, chains them together, and iterates until it has sufficient information to answer the question.

Because every iteration depends on the model emitting valid JSON in tool_calls[i].function.arguments, a single malformed argument can break the entire chain. Add post_processing_steps: [{"type": "json-repair"}] to your request to automatically repair common JSON errors before they reach your loop. See Post-processing.

Getting started

Enable the model to make multiple sequential tool calls:

import requests

headers = {
  "authorization": "<YOUR_API_KEY>"
}

conversation_history = [
    {"role": "user", "content": "Where does Sarah work and what city is that in?"}
]

max_iterations = 10
iteration = 0

while iteration < max_iterations:
    iteration += 1
    response = chat(conversation_history, model)
    choice = response["choices"][0]

    if choice.get("message", {}).get("tool_calls"):
        # Execute tools and add results to history
        handle_tool_calls(choice["message"]["tool_calls"], conversation_history)
        # Continue loop - model can make more tool calls
        continue
    else:
        # Model has final answer
        print(choice["message"]["content"])
        break

Example interaction

You: Where does Andrew live?
🔧 Calling: search_employees(first_name="Andrew")
✅ Result: Found 1 result
🔧 Calling: search_tavily(query="33186 zip code location")
✅ Result: The zip code 33186 is located in Miami, Florida...
Assistant: Andrew lives in Miami, Florida (zip code 33186).

How agentic behavior works

The model autonomously:

Decides which tools to use based on the question
Chains multiple tools together (e.g., get employee info → lookup location details)
Iterates until it has sufficient information to answer
Handles complex multi-step reasoning automatically

This is achieved by running a loop that continues making requests as long as the model is calling tools.

API reference

Request

The LLM Gateway accepts POST requests to https://llm-gateway.assemblyai.com/v1/chat/completions with the following parameters:

curl -X POST \
  "https://llm-gateway.assemblyai.com/v1/chat/completions" \
  -H "Authorization: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-sonnet-4-6",
    "messages": [
      {
        "role": "user",
        "content": "Where does Sarah work and what city is that in?"
      }
    ],
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "search_employees",
          "description": "Search employee database",
          "parameters": {
            "type": "object",
            "properties": {
              "first_name": {
                "type": "string"
              }
            }
          }
        }
      },
      {
        "type": "function",
        "function": {
          "name": "search_web",
          "description": "Search the web for information",
          "parameters": {
            "type": "object",
            "properties": {
              "query": {
                "type": "string"
              }
            }
          }
        }
      }
    ],
    "max_tokens": 1000
  }'

Request parameters

Key	Type	Required?	Description
`model`	string	Yes	The model to use for completion. See Available models section for supported values.
`messages`	array	Yes	An array of message objects representing the conversation history.
`tools`	array	Yes	An array of tool definitions that the model can call.
`tool_choice`	string or object	No	Controls which tools the model can call. Options: `"none"`, `"auto"`, or an object specifying a specific function.
`max_tokens`	number	No	The maximum number of tokens to generate. Default: 1000. Range: [1, context_length).
`temperature`	number	No	Controls randomness in the output. Higher values make output more random. Range: [0, 2].

Message object

Key	Type	Required?	Description
`role`	string	Yes	The role of the message sender. Valid values: `"user"`, `"assistant"`, `"system"`, or `"tool"`.
`content`	string or array	Yes	The message content. Can be a string or an array of content parts for the `"user"` role.
`name`	string	No	An optional name for the message sender. For non-OpenAI models, this will be prepended as `{name}: {content}`.
`tool_call_id`	string	No*	Required when `role` is `"tool"`. The ID of the tool call this message is responding to.

Tool object

Key	Type	Required?	Description
`type`	string	Yes	The type of tool. Currently only `"function"` is supported.
`function`	object	Yes	The function definition.
`function.name`	string	Yes	The name of the function.
`function.description`	string	No	A description of what the function does.
`function.parameters`	object	Yes	A JSON Schema object describing the function parameters.

Tool choice object

The tool_choice parameter can be:

"none" - The model will not call any tools
"auto" - The model can choose whether to call tools
An object with type: "function" and a function.name to force calling a specific function

Response

The API returns a JSON response. In agentic workflows, the model may make multiple tool calls before providing a final answer.

First response (tool call)

{
  "request_id": "abc123",
  "choices": [
    {
      "index": 0,
      "finish_reason": "tool_use",
      "message": {
        "role": "assistant",
        "content": "",
        "tool_calls": [
          {
            "id": "call_123",
            "type": "function",
            "function": {
              "name": "search_employees",
              "arguments": "{\"first_name\": \"Sarah\"}"
            }
          }
        ]
      }
    }
  ],
  "request": { "model": "claude-sonnet-4-6", "max_tokens": 1000 },
  "usage": { "input_tokens": 150, "output_tokens": 30, "total_tokens": 180 }
}

Subsequent response (another tool call)

After adding the function result to conversation history:

{
  "request_id": "def456",
  "choices": [
    {
      "index": 0,
      "finish_reason": "tool_use",
      "message": {
        "role": "assistant",
        "content": "",
        "tool_calls": [
          {
            "id": "call_456",
            "type": "function",
            "function": {
              "name": "search_web",
              "arguments": "{\"query\": \"Acme Corp headquarters location\"}"
            }
          }
        ]
      }
    }
  ],
  "request": { "model": "claude-sonnet-4-6", "max_tokens": 1000 },
  "usage": { "input_tokens": 220, "output_tokens": 35, "total_tokens": 255 }
}

Final response (answer)

After all tool calls are complete:

{
  "request_id": "ghi789",
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "Sarah works at Acme Corp, which is headquartered in San Francisco, California."
      },
      "finish_reason": "stop"
    }
  ],
  "request": {
    "model": "claude-sonnet-4-6",
    "max_tokens": 1000
  },
  "usage": {
    "input_tokens": 280,
    "output_tokens": 20,
    "total_tokens": 300
  }
}

Response fields

Key	Type	Description
`request_id`	string	A unique identifier for the request.
`choices`	array	An array of completion choices. Typically contains one choice.
`choices[i].message`	object	The message object containing the model’s response.
`choices[i].message.role`	string	The role of the message, typically `"assistant"`.
`choices[i].message.content`	string	The text content of the model’s response. May be null when calling tools.
`choices[i].finish_reason`	string	The reason the model stopped generating. Common values: `"stop"`, `"length"`, `"tool_calls"` (OpenAI models), `"tool_use"`, `"end_turn"` (Claude models). LLM Gateway passes the provider-native value through; client code should accept either family.
`choices[i].message.tool_calls`	array	Present on assistant message choices when the model calls a tool. Each tool call appears in its own choice with the calling content nested under `message`.
`request`	object	Echo of the request parameters (excluding `messages`).
`usage`	object	Token usage statistics for the request.
`usage.input_tokens`	number	Number of tokens in the prompt.
`usage.output_tokens`	number	Number of tokens in the completion.
`usage.total_tokens`	number	Total tokens used (prompt + completion).

Tool call object

When the model wants to call a tool, the response includes a tool_calls array:

Key	Type	Description
`tool_calls[i].id`	string	A unique identifier for the tool call.
`tool_calls[i].type`	string	The type of tool call, always `"function"`.
`tool_calls[i].function`	object	The function call details.
`tool_calls[i].function.name`	string	The name of the function to call.
`tool_calls[i].function.arguments`	string	A JSON string containing the function arguments.

Error response

If an error occurs, the API returns an error response:

{
  "code": 400,
  "message": "invalid request body",
  "request_id": "2a9adf03-c73e-4333-a42d-54b515e6afbd",
  "metadata": {
    "errors": [
      "one of messages or prompt required"
    ]
  }
}

Key	Type	Description
`code`	number	HTTP status code for the error.
`message`	string	A human-readable description of the error.
`request_id`	string	Unique identifier for the request. Include this when contacting support.
`metadata`	object	Optional. Present on 400 responses with per-field validation details.
`metadata.errors`	array	List of specific validation failure messages.

Common error codes

Code	Description
400	Bad Request - Invalid request parameters
401	Unauthorized - Invalid or missing API key
403	Forbidden - Insufficient permissions
404	Not Found - Invalid endpoint or model
429	Too Many Requests - Rate limit exceeded
500	Internal Server Error - Server-side error

Getting started

Features

API reference

Core concepts

Voice AI Guides

Overview

Getting started

Example interaction

How agentic behavior works

API reference

Request

Request parameters

Message object

Tool object

Tool choice object

Response

First response (tool call)

Subsequent response (another tool call)

Final response (answer)

Response fields

Tool call object

Error response

Common error codes

​Overview

​Getting started

​Example interaction

​How agentic behavior works

​API reference

​Request

​Request parameters

​Message object

​Tool object

​Tool choice object

​Response

​First response (tool call)

​Subsequent response (another tool call)

​Final response (answer)

​Response fields

​Tool call object

​Error response

​Common error codes

Overview

Getting started

Example interaction

How agentic behavior works

API reference

Request

Request parameters

Message object

Tool object

Tool choice object

Response

First response (tool call)

Subsequent response (another tool call)

Final response (answer)

Response fields

Tool call object

Error response

Common error codes