Build a browser voice assistant in JavaScript with AssemblyAI

A browser voice assistant is a web app that listens through the microphone, understands the request and talks back, all in a browser tab. With AssemblyAI, the whole pipeline runs on one API key: Universal-3.6 Pro Realtime transcribes the user, an LLM writes the reply through LLM Gateway, and the AssemblyAI Text-to-Speech API speaks it. Your JavaScript moves audio bytes and renders transcripts, and a small Node server keeps the key.

Most “JavaScript voice assistant” tutorials stop at the Web Speech API: recognize a phrase, match a command, say something back. That’s fine for “open YouTube.” It falls apart when you want a real conversation, with an LLM, interruptions and natural turn-taking.

This tutorial builds the conversational version: a Node server that mints temporary tokens and calls the LLM, AudioWorklet mic capture that works in Chrome, Firefox and Safari, gapless playback of streamed speech, barge-in and a clean hang-up. It comes to about 350 lines of code, comments included. If you’d rather start in Python, the Python voice agent tutorial builds a voice agent that runs on the server.

Web Speech API vs a streaming voice agent

Browsers ship a built-in option, so it’s worth being clear about when you need more. The Web Speech API has two parts: SpeechRecognition (speech to text) and speechSynthesis (text to speech). Here’s the whole thing in a few lines:

// Web Speech API: recognition + browser TTS, no LLM in between
const SR = window.SpeechRecognition || window.webkitSpeechRecognition;
if (SR) {
  const rec = new SR();
  rec.lang = "en-US";
  rec.onresult = (e) => {
    const said = e.results[0][0].transcript;
    speechSynthesis.speak(new SpeechSynthesisUtterance(`You said: ${said}`));
  };
  rec.start();
} else {
  console.warn("SpeechRecognition isn't supported in this browser.");
}

That works for short commands. But there’s no LLM in the loop, so you’re writing the “understanding” yourself. Recognition support also varies by browser. And by default, recognition uses a service provided by the user’s platform, so you don’t control the model or its accuracy.

The rule of thumb: if the user says a phrase and your app does one thing, use the Web Speech API. If the user has a conversation, use a voice agent.

One vendor for the whole pipeline

A browser voice agent always has three stages: streaming speech-to-text, an LLM and text-to-speech. What you choose is how many vendors sit behind them. Wiring three providers means three keys, three billing accounts and three sets of failure modes to handle in glue code. Running all three on AssemblyAI keeps one key on your server and one set of docs.

Speech-to-text runs at wss://streaming.assemblyai.com/v3/ws with speech_model=universal-3-6-pro, which posts a 5.19% mean word error rate on AssemblyAI’s English voice-agent benchmark and covers 32 languages. Text-to-speech runs at wss://tts.us.assemblyai.com/v1/ws on the assemblyai-tts-1 model, which is built for conversational voice agents and trained on millions of hours of audio, including real conversational data. It delivers frontier-level quality, runs faster and costs less than other production voice agent TTS models, and has unlimited concurrency. It speaks English, Spanish, French, German, Italian and Portuguese. For what happens at each stage, see how a voice agent pipeline works.

Architecture

The app has two halves: a browser client and a small Node server.

1. The browser asks the Node server for two temporary tokens, one for speech-to-text and one for text-to-speech. Your API key stays on the server.
2. The browser opens the speech-to-text WebSocket and the text-to-speech session with those tokens.
3. An AudioWorklet streams mic audio up as 16 kHz PCM.
4. When a turn ends, the browser sends the conversation to the server, which calls the LLM through LLM Gateway and returns the reply.
5. The browser speaks the reply on the text-to-speech session and plays the raw PCM back-to-back with the Web Audio API.

Prerequisites

• Node.js 18+ (native fetch and ES modules)
• A modern browser with AudioWorklet support (current Chrome, Edge, Firefox or Safari)
• An AssemblyAI API key. LLM Gateway may require a paid account; if yours doesn’t have access, point /api/reply at your own LLM provider.

The browser only allows mic access on a secure origin. http://localhost counts, so you can develop locally without TLS. Deploy anywhere else over HTTPS.

Step 1: Set up the project

mkdir voice-assistant && cd voice-assistant
npm init -y
npm pkg set type=module
npm install express dotenv assemblyai
mkdir public
echo "ASSEMBLYAI_API_KEY=your_api_key_here" > .env

Replace the placeholder in .env with your key. You’ll create four files: server.js, public/index.html, public/pcm-worklet.js and public/app.js. The assemblyai package (4.42.0 or later) runs on the server and also ships a browser build, which the server hands to the page.

Step 2: Mint temporary tokens and call the LLM on the server

The browser never sees your API key. The server does three jobs: it serves the SDK’s browser build, it mints a speech-to-text token and a text-to-speech token on /api/tokens, and it calls the LLM on /api/reply. expires_in_seconds is the window in which each token can open its connection. A session that is already open isn’t affected when its token expires.

// server.js: serves the app, mints temporary tokens and calls the LLM.
// Your API key never leaves this process.
import "dotenv/config";
import express from "express";
import { AssemblyAI } from "assemblyai";

const client = new AssemblyAI({ apiKey: process.env.ASSEMBLYAI_API_KEY });
const app = express();
app.use(express.json());
app.use(express.static("public"));
// The SDK's browser build is one self-contained ES module; serve it to the page.
app.get("/vendor/assemblyai.mjs", (req, res) =>
  res.sendFile("node_modules/assemblyai/dist/browser.mjs", { root: process.cwd() })
);

// One call mints both tokens: speech-to-text and text-to-speech each need their own.
app.get("/api/tokens", async (req, res) => {
  try {
    const [stt, tts] = await Promise.all([
      client.streaming.createTemporaryToken({ expires_in_seconds: 60 }),
      client.tts.createTemporaryToken({ expires_in_seconds: 60 }),
    ]);
    res.json({ stt, tts });
  } catch (err) {
    res.status(502).json({ error: err.message });
  }
});

// The LLM call stays on the server too, through LLM Gateway with the same key.
app.post("/api/reply", async (req, res) => {
  const { systemPrompt, messages } = req.body;
  try {
    const completion = await client.llmGateway.chatCompletions({
      model: "qwen3.5-4b-32k-fast", // any model from the LLM Gateway model list
      messages: [{ role: "system", content: systemPrompt }, ...messages],
      max_tokens: 200,
    });
    res.json({ text: completion.choices[0].message.content });
  } catch (err) {
    res.status(502).json({ error: err.message });
  }
});

app.listen(3000, () => console.log("Open http://localhost:3000"));

Mint fresh tokens for every connection, reconnects included, and keep the window short. The model name is an example; pick any model from the LLM Gateway model list. Before you deploy, put authentication and rate limiting in front of both routes, since anyone who can reach them can spend your credits.

Step 3: Capture mic audio with an AudioWorklet

An AudioWorklet runs on its own audio thread, so mic processing doesn’t stutter when the page is busy. It receives Float32 samples, resamples them to 16 kHz, clips them and posts Int16 buffers back to the page.

Why resample instead of creating the AudioContext at 16 kHz? Chrome honors a requested sample rate, but Safari ignores it and usually runs at 48 kHz. Firefox honors it but bypasses its echo canceller at non-default rates. Resampling inside the worklet works the same everywhere, and the page’s AudioContext stays at its native rate for playback.

// public/pcm-worklet.js: converts mic audio to 16 kHz, 16-bit PCM for speech-to-text.
// Runs off the main thread. Resamples from whatever rate the browser uses
// (Safari ignores a requested sampleRate; Firefox loses echo cancellation with one).
const TARGET_RATE = 16000;

class PCMWorklet extends AudioWorkletProcessor {
  constructor() {
    super();
    this.step = sampleRate / TARGET_RATE; // `sampleRate` is a global in worklet scope
    this.pos = 0;                         // fractional read position
    this.carry = new Float32Array(0);
  }

  process(inputs) {
    const input = inputs[0][0];
    if (!input) return true;

    // Join leftover samples from the previous block with the new block.
    const buf = new Float32Array(this.carry.length + input.length);
    buf.set(this.carry);
    buf.set(input, this.carry.length);

    // Linear-interpolation resample, then clip and convert to Int16.
    const out = [];
    while (this.pos + 1 < buf.length) {
      const i = Math.floor(this.pos);
      const frac = this.pos - i;
      const s = Math.max(-1, Math.min(1, buf[i] * (1 - frac) + buf[i + 1] * frac));
      out.push(s < 0 ? s * 0x8000 : s * 0x7fff);
      this.pos += this.step;
    }

    const used = Math.floor(this.pos);
    this.carry = buf.slice(used);
    this.pos -= used;

    if (out.length) {
      const pcm = Int16Array.from(out);
      this.port.postMessage(pcm.buffer, [pcm.buffer]); // transfer, don't copy
    }
    return true;
  }
}

registerProcessor("pcm-worklet", PCMWorklet);

Step 4: Open the speech-to-text socket and the speech session

Create public/app.js. The connect function gets both tokens, builds the audio graph, opens a TextToSpeechSession straight from the SDK’s browser build and opens the speech-to-text WebSocket. Both take the token in place of an API key. Once the speech session is open, the assistant greets the user.

// public/app.js (part 1): tokens, mic, and the two sockets
import { TextToSpeechSession } from "/vendor/assemblyai.mjs";

const STT_URL = "wss://streaming.assemblyai.com/v3/ws";
let stt, tts, audioCtx, micStream, workletNode;
const history = []; // the conversation so far, sent to /api/reply each turn

async function connect() {
  // 1. Fresh temporary tokens from our server, one pair per connection
  const res = await fetch("/api/tokens");
  if (!res.ok) throw new Error(`Token request failed: ${res.status}`);
  const tokens = await res.json();

  // 2. Audio graph at the browser's default rate; the worklet resamples to 16 kHz
  audioCtx = new AudioContext();
  await audioCtx.resume(); // must run inside a click handler
  await audioCtx.audioWorklet.addModule("pcm-worklet.js");
  micStream = await navigator.mediaDevices.getUserMedia({
    audio: { channelCount: 1, echoCancellation: true, noiseSuppression: false },
  });
  workletNode = new AudioWorkletNode(audioCtx, "pcm-worklet");
  audioCtx.createMediaStreamSource(micStream).connect(workletNode);

  // 3. Text-to-speech session, opened with its token: no API key in the page
  tts = new TextToSpeechSession({
    token: tokens.tts,
    voiceId: document.querySelector("#voice").value,
    language: "english",
  });
  await tts.connect(); // resolves with the Begin frame

  // 4. Speech-to-text WebSocket, opened with its own token
  const params = new URLSearchParams({
    speech_model: "universal-3-6-pro",
    encoding: "pcm_s16le",
    sample_rate: "16000",
    token: tokens.stt,
  });
  stt = new WebSocket(`${STT_URL}?${params}`);
  stt.onmessage = (e) => handleEvent(JSON.parse(e.data));
  stt.onclose = (e) => setStatus(`Disconnected (${e.code})`);

  speakReply("Hi! What can I help you with today?");
}

We request echoCancellation: true so the speech-to-text stream doesn’t hear the agent through the speakers and interrupt it. The session’s voice and language are fixed at connect time; to change either, open a new session.

Next, handle speech-to-text events. Mic audio only starts flowing after Begin. Each Turn carries the transcript so far, and end_of_turn: true marks the moment the model decides the user has finished, which is when the app asks the LLM for a reply.

// public/app.js (part 2): react to speech-to-text events
const MIN_WORDS = 2; // shorter turns during agent speech are backchannel ("mhm")
let pending = [];
let pendingSamples = 0;

function handleEvent(evt) {
  switch (evt.type) {
    case "Begin":
      setStatus("Listening");
      // Only start streaming mic audio once the session has begun
      workletNode.port.onmessage = (e) => sendMic(new Int16Array(e.data));
      break;
    case "Turn": {
      const words = evt.transcript.split(/\s+/).filter(Boolean).length;
      if (isSpeaking()) {
        if (words < MIN_WORDS) return; // "mhm" while the agent talks: ignore it
        stopPlayback();                // a real interruption: barge in
      }
      if (!evt.end_of_turn) {
        showPartial(evt.transcript);
      } else if (words > 0) {
        addMessage("You", evt.transcript);
        respond(evt.transcript);
      }
      break;
    }
    case "Termination":
      cleanup();
      break;
  }
}

// Worklet blocks are tiny; batch them into ~100 ms messages (1,600 samples at 16 kHz)
function sendMic(samples) {
  pending.push(samples);
  pendingSamples += samples.length;
  if (pendingSamples < 1600) return;

  const chunk = new Int16Array(pendingSamples);
  let offset = 0;
  for (const s of pending) {
    chunk.set(s, offset);
    offset += s.length;
  }
  pending = [];
  pendingSamples = 0;
  if (stt.readyState === WebSocket.OPEN) stt.send(chunk.buffer); // binary PCM
}

Step 5: Ask the LLM and speak the reply

When a turn ends, respond sends the conversation to /api/reply. The reply goes to speakReply, which calls speak() on the text-to-speech session. speak() sends the text, flushes it and yields the reply’s audio as it’s synthesized, so playback starts with the first chunk rather than at the end of the reply. A turn counter drops a reply that arrives after the user has already said something new.

// public/app.js (part 3): ask the LLM, then speak the reply
let streaming = false; // audio still arriving from the TTS session
let turnId = 0;

async function respond(userText) {
  const myTurn = ++turnId;
  history.push({ role: "user", content: userText });

  const res = await fetch("/api/reply", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({
      systemPrompt: document.querySelector("#prompt").value,
      messages: history,
    }),
  });
  const { text, error } = await res.json();
  if (!res.ok) return setStatus(`LLM error: ${error}`);
  if (myTurn !== turnId) return; // the user spoke again while the LLM was thinking

  await speakReply(text);
}

async function speakReply(text) {
  history.push({ role: "assistant", content: text });
  addMessage("Agent", text);
  streaming = true;
  try {
    // speak() sends the text, flushes it and yields raw PCM chunks.
    // The loop ends at the reply's FlushDone, or at once after cancel().
    for await (const chunk of tts.speak(text)) playChunk(chunk);
  } catch (err) {
    setStatus(`TTS error: ${err.message}`);
  } finally {
    streaming = false;
  }
}

Step 6: Play raw PCM without gaps

The session streams raw 16-bit PCM with no header, so an <audio> element or a Blob plays nothing. Decode the samples yourself: convert each Int16 sample to Float32, copy them into an AudioBuffer at the sample rate the session reported in Begin, and schedule each buffer where the previous one ends. Chunks arrive faster than they play, so calling start(0) on each one would overlap them.

// public/app.js (part 4): gapless playback of raw PCM
let playbackTime = 0;
const scheduled = new Set();

function playChunk(bytes) {
  // 16-bit little-endian PCM at the rate the session reports in Begin (24 kHz by default)
  const rate = tts.begin.configuration.sample_rate;
  const pcm = new Int16Array(bytes.slice().buffer); // copy: Int16Array needs an aligned buffer

  const buffer = audioCtx.createBuffer(1, pcm.length, rate);
  const channel = buffer.getChannelData(0);
  for (let i = 0; i < pcm.length; i++) channel[i] = pcm[i] / 0x8000;

  const src = audioCtx.createBufferSource();
  src.buffer = buffer;
  src.connect(audioCtx.destination);

  playbackTime = Math.max(playbackTime, audioCtx.currentTime);
  src.start(playbackTime); // schedule right after the previous chunk
  playbackTime += buffer.duration;

  scheduled.add(src);
  src.onended = () => scheduled.delete(src);
}

function isSpeaking() {
  return streaming || (audioCtx && playbackTime > audioCtx.currentTime);
}

Step 7: Handle barge-in

When the user talks over the agent, two things have to stop: synthesis on the server and the audio already scheduled in the browser. cancel() handles the first. It stops the request in flight, drops any text not yet spoken and ends the speak() loop at once. Stopping every scheduled source and resetting the cursor handles the second.

// public/app.js (part 5): barge-in
function stopPlayback() {
  tts.cancel(); // stops synthesis and ends the speak() loop
  for (const src of scheduled) {
    try { src.stop(); } catch { /* already finished */ }
  }
  scheduled.clear();
  playbackTime = audioCtx.currentTime;
  turnId++; // drop any LLM reply still on its way
}

The barge-in check in Step 4 ignores turns shorter than two words while the agent is talking, so a “mhm” or “okay” doesn’t cut the agent off. Raise MIN_WORDS if short filler still gets through. The streaming voice agent guide describes a fuller backchannel filter.

Step 8: End the session cleanly

Don’t just close the sockets. Send Terminate on the speech-to-text socket, and the server finalizes the open turn and answers with Termination. Then close() the speech session, which sends its own Terminate and waits for the session summary, and release the mic and audio context.

// public/app.js (part 6): end the session cleanly
function hangUp() {
  if (stt?.readyState === WebSocket.OPEN) {
    stt.send(JSON.stringify({ type: "Terminate" })); // server replies with Termination
  }
}

function cleanup() {
  stt?.close();
  tts?.close().catch(() => {}); // sends Terminate and waits for the summary
  micStream?.getTracks().forEach((t) => t.stop());
  audioCtx?.close();
  setStatus("Session ended");
}

// Tab closed or navigated away
window.addEventListener("pagehide", hangUp);

Step 9: Add the page and UI wiring

<!-- public/index.html -->
<!doctype html>
<html lang="en">
  <head>
    <meta charset="utf-8" />
    <title>Browser voice assistant</title>
  </head>
  <body>
    <label>Voice
      <select id="voice">
        <option value="49c443a4-8a1d-5f7e-b70b-98bdbd9823c3">Audrey (American English)</option>
        <option value="aai-sara">Sara (American English)</option>
        <option value="aai-peter">Peter (American English)</option>
        <option value="aai-nancy">Nancy (British English)</option>
      </select>
    </label>
    <label>System prompt
      <textarea id="prompt">You are a friendly voice assistant. Keep replies to one or two short sentences.</textarea>
    </label>
    <button id="connect">Connect</button>
    <button id="hangup">Hang up</button>
    <p id="status">Idle</p>
    <div id="log"></div>
    <script type="module" src="app.js"></script>
  </body>
</html>

app.js loads as an ES module, so it can import the SDK’s browser build from the server. Add the UI helpers to the end of public/app.js.

// public/app.js (part 7): minimal UI wiring
const statusEl = document.querySelector("#status");
const logEl = document.querySelector("#log");
let partialEl = null;

function setStatus(text) {
  statusEl.textContent = text;
}

function showPartial(text) {
  if (!partialEl) {
    partialEl = document.createElement("p");
    partialEl.className = "partial";
    logEl.append(partialEl);
  }
  partialEl.textContent = text;
}

function addMessage(who, text) {
  if (who === "You" && partialEl) {
    partialEl.remove();
    partialEl = null;
  }
  const p = document.createElement("p");
  p.textContent = `${who}: ${text}`;
  logEl.append(p);
}

document.querySelector("#connect").onclick = () =>
  connect().catch((err) => setStatus(err.message));
document.querySelector("#hangup").onclick = hangUp;

Run node server.js, open http://localhost:3000, click Connect, allow the mic and start talking. The assistant greets you, your words appear as a partial transcript and then a final message, and the agent’s reply plays back.

Customize your assistant

Pick a different voice

The voice dropdown sets voiceId on the text-to-speech session. It accepts a voice’s UUID or its library slug, such as aai-sara. The voices guide lists the built-in voices, and the dashboard catalog holds thousands more across the six languages, each with a Copy voice ID button. To speak Spanish, French, German, Italian or Portuguese, set language to the language’s English name, such as "spanish". Every voice speaks all six. Voice and language are fixed for a session, so set them before you connect.

Change the personality with the system prompt

The system prompt travels with every /api/reply request, so edit the textarea and the next turn uses it. No reconnect needed. In production, keep the prompt on the server and let the page send only the conversation. Keep replies short and in full sentences with punctuation; the text-to-speech model takes its pacing and tone from the punctuation. The accuracy and latency guide has more on writing text that reads well aloud.

Tune turn detection

Universal-3.6 Pro Realtime decides when a turn ends on its own, and its defaults suit most conversations. Start there. If users get cut off while reading out long numbers or email addresses, see the turn detection settings in the streaming voice agent guide before you change anything, and test each change against real calls. On the barge-in side, MIN_WORDS in Step 4 is the knob that matters most.

Call your own functions

Tools live in the LLM request. Pass OpenAI-style tools to LLM Gateway, run the function the model asks for, append the result as a tool message and call the model again. The browser code doesn’t change, since it still receives one reply to speak.

// server.js (optional): give the LLM a tool. Replace the /api/reply route with this one.
const tools = [{
  type: "function",
  function: {
    name: "get_order_status",
    description: "Look up the status of an order by its number",
    parameters: {
      type: "object",
      properties: { order_number: { type: "string" } },
      required: ["order_number"],
    },
  },
}];

async function getOrderStatus({ order_number }) {
  return { order_number, status: "shipped" }; // call your own backend here
}

app.post("/api/reply", async (req, res) => {
  const messages = [
    { role: "system", content: req.body.systemPrompt },
    ...req.body.messages,
  ];
  try {
    let message;
    for (let round = 0; round < 3; round++) {
      const completion = await client.llmGateway.chatCompletions({
        model: "qwen3.5-4b-32k-fast",
        messages,
        tools,
        max_tokens: 200,
      });
      message = completion.choices[0].message;
      if (!message.tool_calls?.length) break;

      // Run each requested tool and hand the results back to the model
      messages.push(message);
      for (const call of message.tool_calls) {
        const result = await getOrderStatus(JSON.parse(call.function.arguments));
        messages.push({ role: "tool", tool_call_id: call.id, content: JSON.stringify(result) });
      }
    }
    res.json({ text: message.content });
  } catch (err) {
    res.status(502).json({ error: err.message });
  }
});

Troubleshooting

Mic blocked or no audio going up. getUserMedia only works on secure origins. http://localhost counts; http://your-laptop.local doesn’t. For cross-device testing, put TLS in front of the Node server (Caddy, Cloudflare Tunnel or ngrok). Also make sure audioCtx.resume() runs inside the click handler.

Agent keeps interrupting itself. That’s acoustic echo: the mic is picking up the speakers. Confirm echoCancellation: true and use headphones while developing.

Transcripts are garbled. It’s a sample-rate mismatch. The speech-to-text URL says sample_rate=16000, so make sure every byte goes through the worklet’s resampler. Don’t send raw samples from a 44.1 or 48 kHz context labeled as 16 kHz.

Replies arrive but nothing plays. The speech session sends raw PCM with no header. Play it through the Web Audio API as in Step 6, at the rate in tts.begin.configuration.sample_rate.

connect() rejects with an unauthorized error, or a socket closes with 1008. The token expired before the connection opened, or it was minted for the wrong product. Speech-to-text and text-to-speech each need their own token. Fetch fresh tokens for every connection and confirm ASSEMBLYAI_API_KEY is set on the server.

The speech session closes after two idle minutes. A text-to-speech session closes after two minutes with no frame in either direction. Call tts.keepAlive() on a timer while the user is quiet, or reconnect with a fresh token when the next reply is ready. The limits page covers session length and rate limits.

Where to go next

You now have a conversational assistant running in a browser tab, with one vendor behind every stage and no API key in the client. From here, the work is product work: the prompt, the tools and the UI.

If your assistant needs to run server-side instead, see the Node.js voice agent with Universal-3.6 Pro Realtime and the Python voice agent tutorial. For text-to-speech on its own, start with the Text-to-Speech quickstart, and see choosing an interface for when HTTP beats a WebSocket session.

Frequently asked questions

How can I use voice recognition in JavaScript?

For simple commands, use the browser’s Web Speech API: create a SpeechRecognition (or webkitSpeechRecognition) object and listen for onresult. For conversational assistants, stream mic audio over a WebSocket to a streaming speech-to-text API such as AssemblyAI’s Universal-3.6 Pro Realtime, which returns partial transcripts and marks the end of each turn, then pass the final transcript to an LLM.

Can I build a voice assistant with only the Web Speech API?

Yes, for command-style assistants. The Web Speech API gives you speech recognition and browser text-to-speech, but no language model, turn detection or interruption handling. You’d have to write the understanding logic yourself, and recognition support varies by browser.

How do I use AssemblyAI from the browser without exposing my API key?

Run a small backend that mints temporary tokens with your key and hands them to the page. The JavaScript SDK does it with client.streaming.createTemporaryToken() for speech-to-text and client.tts.createTemporaryToken() for text-to-speech. The browser passes the speech-to-text token as ?token= on the WebSocket URL and opens the text-to-speech session with new TextToSpeechSession({ token }). Mint a new token for every connection and keep expires_in_seconds short.

Does the AssemblyAI JavaScript SDK work in the browser?

Yes. The assemblyai package (4.42.0 and later) ships a browser build, and its TextToSpeechSession runs in the browser when you open it with a temporary token. In this tutorial the server serves that build to the page as an ES module; a bundler resolves the same build through the package’s browser export.

What audio format does the browser send and receive?

This tutorial sends 16 kHz, 16-bit signed little-endian mono PCM to speech-to-text as binary WebSocket messages, resampled in an AudioWorklet. Text-to-speech returns raw 16-bit PCM at 24 kHz by default. Convert each Int16 sample to Float32 and schedule it with the Web Audio API, using the sample rate the session reports in Begin.

How much does it cost?

Universal-3.6 Pro Realtime streaming speech-to-text costs $0.45/hr. LLM Gateway is billed by the model you choose, and text-to-speech is $15 per 1M characters, billed on input characters. See the pricing page for current rates. New accounts get free credits to build and test.