Puppeteer Mode

In puppeteer mode, the avatar says exactly what you tell it to. You send a line of text, and the avatar speaks it on camera, in its own voice. It doesn't listen to the visitor or come up with answers of its own.

By the end of this guide, you'll have a web page with a companion or a digital twin that reads out the lines your backend sends it.

Overview

Puppeteer mode fits when you already know what the avatar should say, or want to decide it yourself:

Scripted experiences such as guided tours, product walkthroughs, and onboarding.
Your own logic: your backend, your own model, or a rules engine picks each reply.
Approved wording: every line is reviewed before the avatar says it.

Puppeteer sessions run no speech recognition and no language model. Nothing transcribes the visitor, and nothing writes replies. The avatar only says the lines you send with the talk command. If you want the agent to listen and answer on its own, use the default conversation mode.

Before you begin

You need:

An API key on the Cascade architecture. See Create an API key — Architectures. Puppeteer mode doesn't work on Realtime or Other keys.
A companion from the catalog or a custom one (see Companion), or a digital twin.
A backend that can keep your API key secret, and a web page that loads the Web SDK.

Create a puppeteer agent

Create the agent with mode set to puppeteer (API reference):

curl -X POST https://companion-api.napster.com/public/agents \
  -H "X-Api-Key: $NAPSTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "companionId": "comp_abc123",
    "name": "Product Tour",
    "voiceId": "alloy",
    "language": "en",
    "mode": "puppeteer",
    "providerSettings": {}
  }'

{
  "id": "agent_abc123",
  "companionId": "comp_abc123",
  "name": "Product Tour",
  "previewUrl": "https://cdn.napster.com/companions/comp_abc123/preview.jpeg",
  "language": "en",
  "voiceId": "alloy",
  "providerSettings": {
    "turnDetection": {
      "threshold": 0.9,
      "prefix_padding_ms": 400,
      "silence_duration_ms": 500
    },
    "temperature": 1.2
  },
  "disableIdleTimeout": false,
  "useWebSearch": false,
  "mode": "puppeteer",
  "channels": [
    { "type": "websocket" },
    { "type": "webrtc" }
  ],
  "created": 1730700000
}

Parameter	Type	Required	Description
`companionId`	string	Yes	The companion or digital twin that appears on screen.
`name`	string	Yes	A label for the agent.
`voiceId`	string	Conditional	The voice for a catalog or custom companion, e.g. `alloy`. Leave it out for a digital twin, which speaks in its cloned voice. Wait until the twin has finished generating: until then, the API treats it as an ordinary companion and returns `VoiceIdRequired`.
`mode`	string	No	Set to `puppeteer`. Defaults to `conversation`.
`providerSettings`	object	Yes	Pass `{}`. The API fills in defaults, but puppeteer sessions don't use them because no model runs.

To switch an existing agent, send PATCH /public/agents/agent_abc123 with { "mode": "puppeteer" }. An agent that already has a SIP or VoIP channel can't be switched.

Issue connection tokens from your backend

Each visit needs a short-lived connection token. Create it on your backend, where the API key stays secret (API reference):

curl -X POST https://companion-api.napster.com/public/agents/agent_abc123/connections \
  -H "X-Api-Key: $NAPSTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "channelType": "webrtc"
  }'

{
  "token": "eyJhbGci...",
  "connection": {
    "id": "conn_abc123"
  }
}

The session uses the agent's mode, so this request has no mode field. Store connection.id if you want to find the session later.

Expose this call as an endpoint your page can reach. For example, in Node.js with Express:

app.post("/api/avatar-session", async (req, res) => {
  const response = await fetch(
    "https://companion-api.napster.com/public/agents/agent_abc123/connections",
    {
      method: "POST",
      headers: {
        "X-Api-Key": process.env.NAPSTER_API_KEY,
        "Content-Type": "application/json",
      },
      body: JSON.stringify({ channelType: "webrtc" }),
    },
  );
  const { token, connection } = await response.json();
  res.json({ token, sessionId: connection.id });
});

Never call the Napster API from the browser. Anyone who reads your page's source could copy the API key.

Make the avatar speak

On the page, fetch a token, start the Web SDK, and send talk once the session is ready. The SDK drops commands sent before the connection is open, so wait for the first ready state:

import { NapsterCompanionApiSdk } from "@touchcastllc/napster-companion-api";
import "@touchcastllc/napster-companion-api/styles";

const { token } = await fetch("/api/avatar-session", { method: "POST" }).then((r) => r.json());

let ready = false;

const instance = await NapsterCompanionApiSdk.init(token, {
  mountContainer: "#avatar-container",
  features: {
    inactiveTimeout: { enabled: false },
  },
  onData: (message) => {
    if (message.event === "avatar_state_changed" && message.data?.state === "ready" && !ready) {
      ready = true;
      say("intro-1", "Welcome to Acme. Let me show you around the new dashboard.");
    }
  },
});

function say(talkId, text) {
  instance.sendCommand({
    type: "talk",
    data: { talkId, text },
  });
}

The talk command takes these fields:

Field	Type	Required	Description
`talkId`	string	Yes	Your ID for the line. It comes back as `talk_id` on `talk_state_changed`.
`text`	string	Yes	The text for the avatar to speak, exactly as written.
`split`	boolean	No	Speak the text sentence by sentence. Defaults to `true`.

Turn off the SDK's inactivity timeout as shown above. The SDK doesn't count puppeteer lines as activity, so with the default settings it ends the session about 90 seconds after it starts, even while the avatar is talking.

talk isn't in the Web SDK's typed DataChannelCommand union yet. In TypeScript, cast the command: instance.sendCommand({ type: "talk", data } as unknown as DataChannelCommand).

Track, queue, and stop lines

Every line reports its progress in talk_state_changed, with your talkId as talk_id:

{
  "event": "talk_state_changed",
  "data": {
    "talk_id": "intro-1",
    "state": "started",
    "sentence_index": null
  }
}

State	Meaning
`preparing`	The line was received and is being prepared.
`started`	The avatar started speaking.
`ended`	The avatar finished the whole line. Fires once per line.
`canceled`	The line stopped before it finished.

In puppeteer mode, sentence_index is always null and there's no in_progress state, even when split breaks the line into sentences. Use started and ended for the whole line.

The text of each line also arrives in a message_received event with role: "assistant" and your talkId as item_id. With split on, you get one event per sentence, numbered by content_index. You can use these for captions.

To play several lines in order, keep a queue and send the next line when the current one reports ended or canceled:

const queue = [];
let current = null;

function enqueue(talkId, text) {
  queue.push({ talkId, text });
  if (!current) playNext();
}

function playNext() {
  current = queue.shift() ?? null;
  if (current) say(current.talkId, current.text);
}

function onTalkState(data) {
  if (data.talk_id !== current?.talkId) return;
  if (data.state === "ended" || data.state === "canceled") playNext();
}

Call onTalkState(message.data) from onData when message.event is talk_state_changed.

Because nothing listens to the visitor, their speech can't interrupt a line. A line stops only when your client sends { "type": "cancel" }. In the Web SDK, call stopAvatarTalking(), for example from a Skip button:

skipButton.addEventListener("click", () => instance.stopAvatarTalking());

The line then reports canceled.

Drive lines from your backend

The talk command travels over the browser's connection to the avatar, so there is no REST endpoint for it. When your backend decides what the avatar says, push each line to your own page and let the page forward it:

Your backend decides the next line.
It sends the line to the page over your own channel, such as a WebSocket or server-sent events.
The page calls enqueue(talkId, text).

A sketch with server-sent events:

// Page
const events = new EventSource("/api/avatar-lines");
events.addEventListener("line", (event) => {
  const { talkId, text } = JSON.parse(event.data);
  enqueue(talkId, text);
});

// Backend (Express): push a line to a connected page
res.write(`event: line\ndata: ${JSON.stringify({ talkId: "price-2", text: "The Pro plan is 20 dollars a month." })}\n\n`);

Puppeteer sessions don't store a transcript, so keep your own log of each line by talkId if you need a record of what the avatar said.

Using the visitor's speech

The platform doesn't transcribe the visitor in puppeteer mode. To react to what visitors say:

Capture the microphone on your side and transcribe it with your own speech recognition.
Run your logic on the transcript.
Reply with talk.

If you'd rather have the platform listen and reply, use conversation mode instead.

Audio-only, server-side variant

If you don't need video, your server can run the whole session over the WebSocket channel with no browser involved. Create the connection with channelType set to websocket, or call POST /public/ws-connections with mode set to puppeteer. Open the socket, send talk as a JSON frame, and receive the spoken line as audio_received events:

{
  "type": "talk",
  "data": {
    "talkId": "ivr-1",
    "text": "Your order has shipped and arrives on Friday."
  }
}

Keep data flowing while the socket is open. Nothing listens in puppeteer mode, but if your server sends nothing at all, the connection drops after about a minute. Streaming silence with send_audio keeps it open.

Over WebSocket, the audio arrives faster than real time, and ended fires once all of it has been sent, not when it has finished playing. If you play the audio, time the next line from your own playback instead of from ended.

Limits

Area	Behavior in puppeteer mode
API key	Cascade architecture only.
Phone channels	SIP and VoIP aren't available.
Listening	No speech recognition, no language model, and no voice barge-in. A line stops only on `cancel`.
MCP	Not available. Cascade keys don't support MCP servers or connectors.
Idle timeout	A session closes after 3 minutes without a line. Each `talk` counts as activity, but the visitor's speech doesn't. For longer pauses, set `disableIdleTimeout` on the agent.
Memory	Cross-session memory switches on only when you pass `externalClientId` on the connection. Without it, nothing carries over between sessions.
Sessions	Sessions are stored with timing and cost, and the session `mode` field shows `puppeteer`. No transcript is stored.

Errors

A rejected request returns 400 with one of these codes:

Code	When
`UnsupportedSessionMode`	You opened a puppeteer session with a key that isn't on the Cascade architecture, or with a Cascade key that has no Cascade settings.
`TelephonyChannelNotAllowed`	You added a SIP or VoIP channel to a puppeteer agent, or switched an agent that has one to puppeteer mode.
`VoiceIdRequired`	The agent uses a catalog or custom companion and has no `voiceId`.
`ClonedVoiceNotAvailable`	The digital twin's cloned voice isn't ready yet. Retry once the twin has finished processing.
`CompanionV2NotEnabled`	The companion uses the v2 avatar model, which isn't enabled for your organization.

Puppeteer Mode

Overview

Before you begin

Create a puppeteer agent

Issue connection tokens from your backend

Make the avatar speak

Track, queue, and stop lines

Drive lines from your backend

Using the visitor's speech

Audio-only, server-side variant

Limits

Errors

Next steps

Client Commands

Server Events

Digital Twins

Web SDK

On this page