Voice AI PlatformLLM Gateway

Agentic Workflows

Overview

Agentic workflows enable the model to make multiple sequential tool calls for complex multi-step reasoning. The model autonomously decides which tools to use, chains them together, and iterates until it has sufficient information to answer the question.

Getting started

Enable the model to make multiple sequential tool calls:

1import requests
2
3headers = {
4  "authorization": "<YOUR_API_KEY>"
5}
6
7conversation_history = [
8    {"role": "user", "content": "Where does Sarah work and what city is that in?"}
9]
10
11max_iterations = 10
12iteration = 0
13
14while iteration < max_iterations:
15    iteration += 1
16    response = chat(conversation_history, model)
17    choice = response["choices"][0]
18
19    if choice.get("tool_calls"):
20        # Execute tools and add results to history
21        handle_tool_calls(choice["tool_calls"], conversation_history)
22        # Continue loop - model can make more tool calls
23        continue
24    else:
25        # Model has final answer
26        print(choice["message"]["content"])
27        break

Example interaction

You: Where does Andrew live?
🔧 Calling: search_employees(first_name="Andrew")
✅ Result: Found 1 result
🔧 Calling: search_tavily(query="33186 zip code location")
✅ Result: The zip code 33186 is located in Miami, Florida...
Assistant: Andrew lives in Miami, Florida (zip code 33186).

How agentic behavior works

The model autonomously:

  • Decides which tools to use based on the question
  • Chains multiple tools together (e.g., get employee info → lookup location details)
  • Iterates until it has sufficient information to answer
  • Handles complex multi-step reasoning automatically

This is achieved by running a loop that continues making requests as long as the model is calling tools.

API reference

Request

The LLM Gateway accepts POST requests to https://llm-gateway.assemblyai.com/v1/chat/completions with the following parameters:

$curl -X POST \
>  "https://llm-gateway.assemblyai.com/v1/chat/completions" \
>  -H "Authorization: YOUR_API_KEY" \
>  -H "Content-Type: application/json" \
>  -d '{
>    "model": "claude-sonnet-4-5-20250929",
>    "messages": [
>      {
>        "role": "user",
>        "content": "Where does Sarah work and what city is that in?"
>      }
>    ],
>    "tools": [
>      {
>        "type": "function",
>        "function": {
>          "name": "search_employees",
>          "description": "Search employee database",
>          "parameters": {
>            "type": "object",
>            "properties": {
>              "first_name": {
>                "type": "string"
>              }
>            }
>          }
>        }
>      },
>      {
>        "type": "function",
>        "function": {
>          "name": "search_web",
>          "description": "Search the web for information",
>          "parameters": {
>            "type": "object",
>            "properties": {
>              "query": {
>                "type": "string"
>              }
>            }
>          }
>        }
>      }
>    ],
>    "max_tokens": 1000
>  }'

Request parameters

KeyTypeRequired?Description
modelstringYesThe model to use for completion. See Available models section for supported values.
messagesarrayYesAn array of message objects representing the conversation history.
toolsarrayYesAn array of tool definitions that the model can call.
tool_choicestring or objectNoControls which tools the model can call. Options: "none", "auto", or an object specifying a specific function.
max_tokensnumberNoThe maximum number of tokens to generate. Range: [1, context_length).
temperaturenumberNoControls randomness in the output. Higher values make output more random. Range: [0, 2].

Message object

KeyTypeRequired?Description
rolestringYesThe role of the message sender. Valid values: "user", "assistant", "system", or "tool".
contentstring or arrayYesThe message content. Can be a string or an array of content parts for the "user" role.
namestringNoAn optional name for the message sender. For non-OpenAI models, this will be prepended as {name}: {content}.
tool_call_idstringNo*Required when role is "tool". The ID of the tool call this message is responding to.

Tool object

KeyTypeRequired?Description
typestringYesThe type of tool. Currently only "function" is supported.
functionobjectYesThe function definition.
function.namestringYesThe name of the function.
function.descriptionstringNoA description of what the function does.
function.parametersobjectYesA JSON Schema object describing the function parameters.

Tool choice object

The tool_choice parameter can be:

  • "none" - The model will not call any tools
  • "auto" - The model can choose whether to call tools
  • An object with type: "function" and a function.name to force calling a specific function

Response

The API returns a JSON response. In agentic workflows, the model may make multiple tool calls before providing a final answer.

First response (tool call)

1{
2  "request_id": "abc123",
3  "choices": [
4    {
5      "message": {
6        "role": "assistant",
7        "content": null
8      },
9      "finish_reason": "tool_calls",
10      "tool_calls": [
11        {
12          "id": "call_123",
13          "type": "function",
14          "function": {
15            "name": "search_employees",
16            "arguments": "{\"first_name\": \"Sarah\"}"
17          }
18        }
19      ]
20    }
21  ],
22  "request": {
23    "model": "claude-sonnet-4-5-20250929",
24    "max_tokens": 1000
25  },
26  "usage": {
27    "prompt_tokens": 150,
28    "completion_tokens": 30,
29    "total_tokens": 180
30  }
31}

Subsequent response (another tool call)

After adding the function result to conversation history:

1{
2  "request_id": "def456",
3  "choices": [
4    {
5      "message": {
6        "role": "assistant",
7        "content": null
8      },
9      "finish_reason": "tool_calls",
10      "tool_calls": [
11        {
12          "id": "call_456",
13          "type": "function",
14          "function": {
15            "name": "search_web",
16            "arguments": "{\"query\": \"Acme Corp headquarters location\"}"
17          }
18        }
19      ]
20    }
21  ],
22  "request": {
23    "model": "claude-sonnet-4-5-20250929",
24    "max_tokens": 1000
25  },
26  "usage": {
27    "prompt_tokens": 220,
28    "completion_tokens": 35,
29    "total_tokens": 255
30  }
31}

Final response (answer)

After all tool calls are complete:

1{
2  "request_id": "ghi789",
3  "choices": [
4    {
5      "message": {
6        "role": "assistant",
7        "content": "Sarah works at Acme Corp, which is headquartered in San Francisco, California."
8      },
9      "finish_reason": "stop"
10    }
11  ],
12  "request": {
13    "model": "claude-sonnet-4-5-20250929",
14    "max_tokens": 1000
15  },
16  "usage": {
17    "prompt_tokens": 280,
18    "completion_tokens": 20,
19    "total_tokens": 300
20  }
21}

Response fields

KeyTypeDescription
request_idstringA unique identifier for the request.
choicesarrayAn array of completion choices. Typically contains one choice.
choices[i].messageobjectThe message object containing the model’s response.
choices[i].message.rolestringThe role of the message, typically "assistant".
choices[i].message.contentstringThe text content of the model’s response. May be null when calling tools.
choices[i].finish_reasonstringThe reason the model stopped generating. Common values: "stop", "length", "tool_calls".
choices[i].tool_callsarrayPresent when the model wants to call tools. Contains function call objects.
requestobjectEcho of the request parameters (excluding messages).
usageobjectToken usage statistics for the request.
usage.prompt_tokensnumberNumber of tokens in the prompt.
usage.completion_tokensnumberNumber of tokens in the completion.
usage.total_tokensnumberTotal tokens used (prompt + completion).

Tool call object

When the model wants to call a tool, the response includes a tool_calls array:

KeyTypeDescription
tool_calls[i].idstringA unique identifier for the tool call.
tool_calls[i].typestringThe type of tool call, always "function".
tool_calls[i].functionobjectThe function call details.
tool_calls[i].function.namestringThe name of the function to call.
tool_calls[i].function.argumentsstringA JSON string containing the function arguments.

Error response

If an error occurs, the API returns an error response:

1{
2  "error": {
3    "code": 400,
4    "message": "Invalid request: missing required field 'model'",
5    "metadata": {}
6  }
7}
KeyTypeDescription
errorobjectContainer for error information.
error.codenumberHTTP status code for the error.
error.messagestringA human-readable description of the error.
error.metadataobjectOptional additional error context.

Common error codes

CodeDescription
400Bad Request - Invalid request parameters
401Unauthorized - Invalid or missing API key
403Forbidden - Insufficient permissions
404Not Found - Invalid endpoint or model
429Too Many Requests - Rate limit exceeded
500Internal Server Error - Server-side error