Inline session configuration

Most agents should create a stored agent instead. Storing your configuration server-side keeps secrets off the client, runs HTTP tools for you, and lets you deploy the same agent across the API, a browser, and Twilio by agent_id. This page covers the inline alternative: configuring an agent at connect-time without storing it, useful for fully dynamic or one-off agents. The field reference below applies to both: the same fields exist on a stored agent’s input/output objects.

session.update is the configuration surface for an inline (non-stored) agent. Everything that shapes how the agent sounds and behaves lives here: its personality, its greeting, the tools it can call, how it interprets user speech, and what voice it speaks in. Send a session.update as your first WebSocket message, and any time after, to update most fields. Some fields can only be set in the first update; see Mutability after session.ready below.

To use a stored agent instead, send { "agent_id": "<id>" } as the only field in your first session.update; it’s mutually exclusive with the inline fields below. See Deploy your agent.

Here’s a full configuration showing every available field:

{
  "type": "session.update",
  "session": {
    "system_prompt": "You are a friendly support agent. Keep responses under 2 sentences.",
    "greeting": "Hi! How can I help you today?",
    "tools": [],
    "input": {
      "format": { "encoding": "audio/pcm" },
      "keyterms": ["AssemblyAI", "Universal"],
      "turn_detection": {
        "vad_threshold": 0.5,
        "min_silence": 1000,
        "max_silence": 3000,
        "interrupt_response": true
      }
    },
    "output": {
      "voice": "ivy",
      "format": { "encoding": "audio/pcm" },
      "volume": 100
    }
  }
}

Every field is optional. Include only what you want to set or change. Jump to any section below for details.

Mutability after `session.ready`

The first session.update you send before session.ready initializes the session. After session.ready, only a subset of fields can be changed. Changing one of the immutable fields raises a session.error with code immutable_field and the rejected change is ignored.

Field	Mutable after `session.ready`?
`session.system_prompt`	Yes. Send a new prompt at any time to change the agent’s behavior on the next turn.
`session.input.turn_detection`	Yes. Adjust VAD thresholds, silence windows, and barge-in on the fly.
`session.input.keyterms`	Yes. Replace the keyterms list at any time. The new list takes effect on the next user utterance.
`session.output.volume`	Yes. Adjust playback volume on the fly.
`session.greeting`	No. Raises `immutable_field`. The greeting is spoken once at session start.
`session.output.voice`	No. Raises `immutable_field`. The voice is bound to the TTS connection at session start.
`session.output.format`	No. Raises `immutable_field`. The output audio encoding is fixed for the session.

Other fields (session.tools, session.input.format) are also accepted in subsequent session.update messages and don’t raise immutable_field.

Fields

The session object carries the same fields as a stored agent. Each has a dedicated guide with details and examples; the cURL there maps one-to-one onto the session.update shape shown above.

system_prompt — the agent’s personality and behavior. Updatable mid-session. See the Prompting guide.
greeting — the exact words spoken on connect, sent straight to TTS. Immutable after session.ready. See Greeting.
output.voice — the TTS voice, e.g. "ivy". Immutable after session.ready. See Voices.
input.format / output.format — audio encoding (default audio/pcm at 24 kHz). output.format is immutable after session.ready. See Audio encoding.
output.volume — playback volume, 0–100. Updatable mid-session. See Volume.
input.keyterms — up to 100 transcription-bias terms. Updatable mid-session. See Key terms.
input.turn_detection — VAD tuning and barge-in. Updatable mid-session. See Turn detection and interruptions.
tools — actions the agent can call. See Add tools.

Getting started

Create & manage agents

Agent behavior

Conversational experience

Deploy

After a session

Reference

API reference

Inline session configuration

Mutability after `session.ready`

Fields

​Mutability after session.ready

​Fields

Mutability after `session.ready`

Fields