Client Commands

Your client can send two commands to the server during an active WebRTC or WebSocket session: send_message to send information to the agent, and set_settings to update session configuration on the fly.

Sending commands

WebRTC (Web SDK)

instance.sendCommand({
  type: "send_message",
  data: {
    role: "user",
    text: "Hello",
    trigger_response: true,
    delay: false
  }
});

WebSocket

Send the command as JSON over the WebSocket connection:

ws.send(JSON.stringify({
  type: "send_message",
  data: {
    role: "user",
    text: "Hello",
    trigger_response: true,
    delay: false
  }
}));

`send_message`

Send text to the agent as a user message or a system context update.

{
  "type": "send_message",
  "data": {
    "role": "user",
    "text": "What is the status of my order?",
    "trigger_response": true,
    "delay": false
  }
}

Parameters

Parameter	Type	Required	Description
`role`	`user` or `system`	Yes	Who the message is from
`text`	string	Yes	The message content
`trigger_response`	boolean	Yes	When `true`, the agent responds immediately. When `false`, the agent absorbs the information silently and may or may not act on it based on conversation context
`delay`	boolean	No	When `true`, waits for the agent to finish speaking before delivering the message and triggering a response. When `false`, delivers immediately — interrupting the agent if it is speaking. Default: `false`

User messages

When role is user, the agent treats the text as if the user typed it instead of speaking. This is useful for text-based input alongside audio, or for injecting user intent programmatically.

{
  "type": "send_message",
  "data": {
    "role": "user",
    "text": "Tell me more about the premium plan",
    "trigger_response": true
  }
}

System messages

When role is system, the text is treated as a context update. The agent incorporates the information into its context without displaying it to the user. Use this when something changes in your application that the agent needs to know about.

{
  "type": "send_message",
  "data": {
    "role": "system",
    "text": "The user just added 'Wireless Headphones' to their shopping cart.",
    "trigger_response": false
  }
}

Triggering a response

When trigger_response is true, the agent generates a response immediately. When false, the agent absorbs the information silently — it does not respond right away, but may incorporate it into future replies based on the conversation context.

The delay parameter controls when the message is delivered. When delay is false (the default), the message is delivered immediately, interrupting the agent if it is currently speaking. When delay is true, the message is held until the agent finishes its current speech, then delivered — and if trigger_response is true, the agent responds after that.

{
  "type": "send_message",
  "data": {
    "role": "system",
    "text": "The user's subscription has been upgraded to premium.",
    "trigger_response": true,
    "delay": true
  }
}

`set_settings`

Update session parameters in real time without creating a new connection. Change the agent's instructions, temperature, or turn detection thresholds.

{
  "type": "set_settings",
  "data": {
    "instructions": "You are a customer support agent. Be concise and helpful.",
    "temperature": 0.7,
    "turn_detection": {
      "threshold": 0.5,
      "prefix_padding_ms": 300,
      "silence_duration_ms": 500
    }
  }
}

All fields are optional. Include only the settings you want to change — omitted fields keep their current values.

Parameters

Parameter	Type	Provider	Description
`instructions`	string	Azure OpenAI	Replace the agent's system instructions
`temperature`	number	Azure OpenAI	Set the response temperature (0-1)
`turn_detection`	object	Azure OpenAI	Configure voice activity detection

`turn_detection`

Parameter	Type	Description
`threshold`	number	Sensitivity threshold for detecting speech
`prefix_padding_ms`	number	Milliseconds of audio to include before detected speech
`silence_duration_ms`	number	Milliseconds of silence before a turn is considered complete

Common patterns

Replacing instructions mid-conversation — instructions fully replaces the agent's system prompt. Use this when the agent's entire role or behavior needs to change, for example when the user moves to a different part of your application:

{
  "type": "set_settings",
  "data": {
    "instructions": "You are a checkout assistant for an e-commerce store. Guide the user through completing their purchase. Confirm the shipping address, suggest available shipping options, and answer questions about return policies. Do not suggest additional products."
  }
}

This replaces the full system prompt — it is not appended to the existing instructions. If you want to provide a contextual update without changing the agent's instructions, use send_message with role: system instead.

Providing context updates — use send_message with role: system to give the agent situational context without replacing its instructions. For example, notify the agent when the user navigates to a different page:

{
  "type": "send_message",
  "data": {
    "role": "system",
    "text": "The user just navigated to the checkout page.",
    "trigger_response": false
  }
}

Adjusting turn detection for noisy environments — increase silence_duration_ms so the agent waits longer before treating silence as the end of the user's turn:

{
  "type": "set_settings",
  "data": {
    "turn_detection": {
      "silence_duration_ms": 1000
    }
  }
}

Client Commands

On this page