> ## Documentation Index > Fetch the complete documentation index at: https://assemblyai.com/docs/llms.txt > Use this file to discover all available pages before exploring further. # Tool Calling ## Overview Tool calling allows you to define functions that the model can call to access external data or perform actions. This enables the model to interact with databases, APIs, and other external systems. Tool call objects follow an OpenAI-compatible structure nested under `choices[i].message.tool_calls`, so the same parser works across Claude, OpenAI, and Gemini. `finish_reason` is passed through provider-native (`tool_use` / `end_turn` for Claude, `tool_calls` / `stop` for OpenAI), so handle both families when branching on it. ## Defining tools Define functions the model can call to access external data or functionality: ```python expandable theme={null} import requests headers = { "authorization": "" } tools = [ { "type": "function", "function": { "name": "search_employees", "description": "Search employee database by name, department, or other criteria", "parameters": { "type": "object", "properties": { "first_name": { "type": "string", "description": "Employee's first name" }, "department": { "type": "string", "description": "Department name" } }, "required": [] } } } ] response = requests.post( "https://llm-gateway.assemblyai.com/v1/chat/completions", headers = headers, json={ "model": "claude-sonnet-4-6", "messages": [{"role": "user", "content": "Find employees in Engineering"}], "tools": tools } ) result = response.json() print(result["choices"][0]["message"]["content"]) ``` ## Handling tool calls Execute tools and send results back to continue the conversation: ```python expandable theme={null} import requests import json headers = { "authorization": "" } choice = response.json()["choices"][0] if choice.get("message", {}).get("tool_calls"): tool_call = choice["message"]["tool_calls"][0] # Add assistant message with tool calls to history conversation_history.append({ "role": "assistant", "content": None, "tool_calls": [tool_call] }) # Execute your function result = execute_function( tool_call["function"]["name"], json.loads(tool_call["function"]["arguments"]) ) # Add tool result to history conversation_history.append({ "role": "tool", "tool_call_id": tool_call["id"], "content": json.dumps({"result": result}) }) # Continue conversation with the result response = requests.post( "https://llm-gateway.assemblyai.com/v1/chat/completions", headers = headers, json = { "model": "claude-sonnet-4-6", "messages": conversation_history, "tools": tools } ) result = response.json() print(result["choices"][0]["message"]["content"]) ``` ## Tool call flow The complete tool calling process: 1. Send user message with `tools` array defining available functions 2. Model responds with `tool_calls` if it needs to use a tool 3. Execute the tool in your application code 4. Add the assistant message (with `tool_calls`) and a `tool` message (with the result) to conversation history 5. Send updated history back to the API 6. Model incorporates results and either calls more tools or provides final answer ## Message types When using tools, structure your conversation history with these message types to track the complete interaction flow: * **user** - Messages from the user * **assistant** - Messages from the AI model * **system** - System instructions or context * **tool** - Results from executing a tool call ## API reference ### Request The LLM Gateway accepts POST requests to `https://llm-gateway.assemblyai.com/v1/chat/completions` with the following parameters: ```bash expandable theme={null} curl -X POST \ "https://llm-gateway.assemblyai.com/v1/chat/completions" \ -H "Authorization: YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "claude-sonnet-4-6", "messages": [ { "role": "user", "content": "Find employees in Engineering" } ], "tools": [ { "type": "function", "function": { "name": "search_employees", "description": "Search employee database by name, department, or other criteria", "parameters": { "type": "object", "properties": { "department": { "type": "string", "description": "Department name" } } } } } ], "max_tokens": 1000 }' ``` #### Request parameters | Key | Type | Required? | Description | | ------------- | ---------------- | --------- | -------------------------------------------------------------------------------------------------------------------- | | `model` | string | Yes | The model to use for completion. See [Available models](/llm-gateway#available-models) section for supported values. | | `messages` | array | Yes | An array of message objects representing the conversation history. | | `tools` | array | Yes | An array of tool definitions that the model can call. | | `tool_choice` | string or object | No | Controls which tools the model can call. Options: `"none"`, `"auto"`, or an object specifying a specific function. | | `max_tokens` | number | No | The maximum number of tokens to generate. Default: 1000. Range: \[1, context\_length). | | `temperature` | number | No | Controls randomness in the output. Higher values make output more random. Range: \[0, 2]. | #### Message object | Key | Type | Required? | Description | | -------------- | --------------- | --------- | -------------------------------------------------------------------------------------------------------------- | | `role` | string | Yes | The role of the message sender. Valid values: `"user"`, `"assistant"`, `"system"`, or `"tool"`. | | `content` | string or array | Yes | The message content. Can be a string or an array of content parts for the `"user"` role. | | `name` | string | No | An optional name for the message sender. For non-OpenAI models, this will be prepended as `{name}: {content}`. | | `tool_call_id` | string | No\* | Required when `role` is `"tool"`. The ID of the tool call this message is responding to. | #### Tool object | Key | Type | Required? | Description | | ---------------------- | ------ | --------- | ----------------------------------------------------------- | | `type` | string | Yes | The type of tool. Currently only `"function"` is supported. | | `function` | object | Yes | The function definition. | | `function.name` | string | Yes | The name of the function. | | `function.description` | string | No | A description of what the function does. | | `function.parameters` | object | Yes | A JSON Schema object describing the function parameters. | #### Tool choice object The `tool_choice` parameter can be: * `"none"` - The model will not call any tools * `"auto"` - The model can choose whether to call tools * An object with `type: "function"` and a `function.name` to force calling a specific function ### Response The API returns a JSON response. When the model wants to call a tool: ```json expandable theme={null} { "request_id": "abc123", "choices": [ { "index": 0, "finish_reason": "tool_use", "message": { "role": "assistant", "content": "I'll search the engineering department." } }, { "index": 1, "finish_reason": "tool_use", "message": { "role": "assistant", "content": "", "tool_calls": [ { "id": "call_123", "type": "function", "function": { "name": "search_employees", "arguments": "{\"department\": \"Engineering\"}" } } ] } } ], "request": { "model": "claude-sonnet-4-6", "max_tokens": 1000 }, "usage": { "input_tokens": 120, "output_tokens": 25, "total_tokens": 145 } } ``` #### Response fields | Key | Type | Description | | ------------------------------- | ------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `request_id` | string | A unique identifier for the request. | | `choices` | array | An array of completion choices. Typically contains one choice. | | `choices[i].message` | object | The message object containing the model's response. | | `choices[i].message.role` | string | The role of the message, typically `"assistant"`. | | `choices[i].message.content` | string | The text content of the model's response. May be null when calling tools. | | `choices[i].finish_reason` | string | The reason the model stopped generating. Common values: `"stop"`, `"length"`, `"tool_calls"` (OpenAI models), `"tool_use"`, `"end_turn"` (Claude models). LLM Gateway passes the provider-native value through; client code should accept either family. | | `choices[i].message.tool_calls` | array | Present on assistant message choices when the model calls a tool. Each tool call appears in its own choice with the calling content nested under `message`. | | `request` | object | Echo of the request parameters (excluding `messages`). | | `usage` | object | Token usage statistics for the request. | | `usage.input_tokens` | number | Number of tokens in the prompt. | | `usage.output_tokens` | number | Number of tokens in the completion. | | `usage.total_tokens` | number | Total tokens used (prompt + completion). | #### Tool call object When the model wants to call a tool, the response includes a `tool_calls` array: | Key | Type | Description | | ---------------------------------- | ------ | ------------------------------------------------ | | `tool_calls[i].id` | string | A unique identifier for the tool call. | | `tool_calls[i].type` | string | The type of tool call, always `"function"`. | | `tool_calls[i].function` | object | The function call details. | | `tool_calls[i].function.name` | string | The name of the function to call. | | `tool_calls[i].function.arguments` | string | A JSON string containing the function arguments. | Models occasionally return malformed JSON in `tool_calls[i].function.arguments`. Add `post_processing_steps: [{"type": "json-repair"}]` to your request to automatically repair common JSON errors. See [Post-processing](/llm-gateway/structured-outputs#post-processing). ### Error response If an error occurs, the API returns an error response: ```json theme={null} { "code": 400, "message": "invalid request body", "request_id": "2a9adf03-c73e-4333-a42d-54b515e6afbd", "metadata": { "errors": [ "one of messages or prompt required" ] } } ``` | Key | Type | Description | | ----------------- | ------ | ------------------------------------------------------------------------ | | `code` | number | HTTP status code for the error. | | `message` | string | A human-readable description of the error. | | `request_id` | string | Unique identifier for the request. Include this when contacting support. | | `metadata` | object | Optional. Present on 400 responses with per-field validation details. | | `metadata.errors` | array | List of specific validation failure messages. | #### Common error codes | Code | Description | | ---- | ----------------------------------------- | | 400 | Bad Request - Invalid request parameters | | 401 | Unauthorized - Invalid or missing API key | | 403 | Forbidden - Insufficient permissions | | 404 | Not Found - Invalid endpoint or model | | 429 | Too Many Requests - Rate limit exceeded | | 500 | Internal Server Error - Server-side error | ## Handling malformed tool call arguments LLMs occasionally return malformed JSON in tool call arguments. Add `post_processing_steps` to your request to automatically repair these before they reach your application: ```json theme={null} { "post_processing_steps": [{"type": "json-repair"}] } ``` See [Post-processing](/llm-gateway/structured-outputs#post-processing) for details.