Voice Agent API

Tool calling

Register tools and handle tool.call / tool.result events so your agent can take real-world actions.

Give your agent the ability to look up information, call APIs, or trigger workflows by registering tools in session.tools. The agent decides when to invoke them and emits a tool.call; you execute the tool and reply with a tool.result — but wait until reply.done before sending it back.

Registering tools

Register tools by passing an array of tool definitions in session.tools on a session.update event. Each tool uses a flat format with type, name, description, and parameters at the top level.

1{
2  "type": "session.update",
3  "session": {
4    "system_prompt": "You are a helpful assistant. Use get_weather for weather questions.",
5    "greeting": "Hi! How can I help?",
6    "tools": [
7      {
8        "type": "function",
9        "name": "get_weather",
10        "description": "Get the current weather for a city.",
11        "parameters": {
12          "type": "object",
13          "properties": {
14            "location": {
15              "type": "string",
16              "description": "City name, e.g. London"
17            }
18          },
19          "required": ["location"]
20        }
21      }
22    ]
23  }
24}

You can update session.tools mid-session by sending another session.update — the new array replaces the previous one.

Handling tool calls

The key pattern: accumulate tool results, then send them all in reply.done — not immediately in tool.call. The agent speaks a transition phrase while waiting; sending results too early can cause timing issues.

1pending_tools: list[dict] = []
2
3# In your event loop:
4
5if t == "tool.call":
6    name = event["name"]
7    args = event.get("args", {})   # args is a plain dict
8
9    # Execute your tool
10    if name == "get_weather":
11        result = {"temp_c": 22, "description": "Sunny"}
12    else:
13        result = {"error": "Unknown tool"}
14
15    # Accumulate — don't send yet
16    pending_tools.append({
17        "call_id": event["call_id"],
18        "result": result,
19    })
20
21elif t == "reply.done":
22    if event.get("status") == "interrupted":
23        # User barged in — discard pending results
24        pending_tools.clear()
25    elif pending_tools:
26        # Now send all tool results
27        for tool in pending_tools:
28            await ws.send(json.dumps({
29                "type": "tool.result",
30                "call_id": tool["call_id"],
31                "result": json.dumps(tool["result"]),
32            }))
33        pending_tools.clear()

If a tool call arrives and the user then interrupts the agent before reply.done completes normally, discard the pending tool results and wait for the next turn. Sending stale results can confuse the agent’s state.