New Voice Standard Bench: 50+ LLMs ranked on real voice tasks Learn more
Insights & Use Cases

Speech recognition in the browser using the Web Speech API

How to use the Web Speech API for browser speech recognition, where it breaks in production, and when to move to a streaming speech-to-text API.

Abstract green diamond illustration

Written by

Patrick Loeber

Published on

29 September 2026

You can add speech recognition to a web page in about fifteen lines of JavaScript. No API key, no backend, no dependency. The Web Speech API has been sitting in browsers since 2013, and for a demo, a hobby project, or an accessibility affordance on a form field, it's genuinely the right answer.

It's also the thing developers most often try first and abandon second.

This guide covers both halves honestly. You'll get working JavaScript speech recognition code you can paste into an HTML file and run immediately — and then a specific, unsentimental account of where the Web Speech API stops working, which browsers silently do something different from what you expected, and what to reach for when the browser API runs out of road.

What is the Web Speech API?

The Web Speech API is a browser standard with two halves that have almost nothing to do with each other:

  • SpeechRecognition — speech-to-text. Takes microphone audio and gives you a transcript.
  • SpeechSynthesis — text-to-speech. Takes a string and speaks it aloud.

This guide is about the first one. SpeechSynthesis is well supported and boring in the good way; SpeechRecognition is where everything interesting and everything frustrating lives.

The key thing to understand up front is that SpeechRecognition is an interface, not an engine. The spec describes how your JavaScript talks to a speech recognizer. It says nothing about what that recognizer is, where it runs, or how accurate it is. In Chrome, your audio is sent to Google's servers. In Safari, it goes to Apple's. The browser is a middleman handing your users' microphone audio to a third party you didn't choose and can't configure.

That single fact explains most of the limitations later in this article.

How the Web Speech API works

The flow is event-driven and short:

  1. You create a SpeechRecognition object and configure it.
  2. You call .start(), which triggers the browser's microphone permission prompt.
  3. The browser streams audio to its recognition service.
  4. You receive result events containing transcript hypotheses, marked either interim (still changing) or final (settled).
  5. Recognition stops — because you called .stop(), because the user went quiet, or because the browser decided to.

Two configuration flags shape everything:

continuous controls whether recognition keeps going after the first final result. Set to false (the default), you get one utterance and the session ends — fine for a voice search box. Set to true, it keeps listening.

interimResults controls whether you see hypotheses as they form. With it off, text appears in one lump after the speaker pauses. With it on, you get the live, word-by-word effect people expect from dictation, with words changing under the cursor as the recognizer revises itself.

You also need the vendor prefix. Chrome, Edge, and Safari all expose this as webkitSpeechRecognition, so every real-world implementation starts with the same line:

const SpeechRecognition = window.SpeechRecognition || window.webkitSpeechRecognition;

A complete speech recognition example

Here's a full, working page. Save it as index.html and open it — with one caveat covered right after the code.

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8" />
    <title>Web Speech API demo</title>
    <style>
      body { font-family: system-ui, sans-serif; max-width: 42rem; margin: 3rem auto; padding: 0 1rem; }
      #transcript { min-height: 8rem; border: 1px solid #ccc; padding: 1rem; margin-top: 1rem; }
      .interim { color: #888; }
      button { font-size: 1rem; padding: 0.5rem 1rem; margin-right: 0.5rem; }
    </style>
  </head>
  <body>
    <h1>Browser speech recognition</h1>

    <button id="start">Start listening</button>
    <button id="stop" disabled>Stop</button>
    <p id="status">Idle.</p>

    <div id="transcript"></div>

    <script>
      const SpeechRecognition =
        window.SpeechRecognition || window.webkitSpeechRecognition;

      const startBtn = document.getElementById("start");
      const stopBtn = document.getElementById("stop");
      const statusEl = document.getElementById("status");
      const transcriptEl = document.getElementById("transcript");

      if (!SpeechRecognition) {
        statusEl.textContent =
          "This browser does not support the Web Speech API.";
        startBtn.disabled = true;
      } else {
        const recognition = new SpeechRecognition();

        recognition.continuous = true;
        recognition.interimResults = true;
        recognition.lang = "en-US";
        recognition.maxAlternatives = 1;

        // Text that has been finalized and will not change again.
        let finalText = "";
        // Set when the user pressed Stop, so we don't auto-restart.
        let userStopped = false;

        recognition.onstart = () => {
          statusEl.textContent = "Listening…";
          startBtn.disabled = true;
          stopBtn.disabled = false;
        };

        recognition.onresult = (event) => {
          let interim = "";

          for (let i = event.resultIndex; i < event.results.length; i++) {
            const result = event.results[i];
            const text = result[0].transcript;

            if (result.isFinal) {
              finalText += text + " ";
            } else {
              interim += text;
            }
          }

          transcriptEl.innerHTML =
            finalText + '<span class="interim">' + interim + "</span>";
        };

        recognition.onerror = (event) => {
          // "no-speech" and "aborted" are routine, not failures.
          if (event.error === "no-speech" || event.error === "aborted") return;

          statusEl.textContent = "Error: " + event.error;

          if (event.error === "not-allowed") {
            statusEl.textContent =
              "Microphone permission denied. Check your browser settings.";
            userStopped = true;
          }
        };

        recognition.onend = () => {
          // Chrome ends the session on its own, roughly every minute and after
          // silence, even with continuous = true. Restart unless the user asked
          // us to stop.
          if (!userStopped) {
            recognition.start();
            return;
          }

          statusEl.textContent = "Stopped.";
          startBtn.disabled = false;
          stopBtn.disabled = true;
        };

        startBtn.addEventListener("click", () => {
          userStopped = false;
          finalText = "";
          transcriptEl.textContent = "";
          recognition.start();
        });

        stopBtn.addEventListener("click", () => {
          userStopped = true;
          recognition.stop();
        });
      }
    </script>
  </body>
</html>

A few things in there aren't obvious from the spec.

The onend restart loop is not optional. Chrome terminates recognition sessions on its own, regardless of continuous = true — after a stretch of silence, and periodically even during continuous speech. If you want a session that outlives a minute, you restart it in onend. Every long-running Web Speech API implementation on the web has some version of this loop.

resultIndex matters. event.results is cumulative: it contains every result from the session, not just the new one. Iterating from event.resultIndex instead of 0 is what keeps you from re-appending text you already have. Skipping this is the single most common bug in Web Speech API code online.

Errors aren't all errors. no-speech fires whenever someone pauses for a few seconds. aborted fires on a normal stop. Surfacing those to users produces an app that looks broken while working correctly.

You need a secure context. file:// won't work in Chrome. Serve it over https:// or from localhost — npx serve or python -m http.server is enough for local work.

Compare Browser And API Transcription

Run the same audio through a production streaming model and hear the difference in accuracy on names, numbers, and noisy rooms. No code required.

Try playground

Browser support and compatibility

This is the section to read before you plan anything around the Web Speech API, because "supported" means far less here than it does for most web platform features.

BrowserSpeechRecognitionWhat you should know
Chrome (desktop)Yes, prefixedThe reference implementation. Audio goes to Google's servers. Sessions are force-ended periodically.
EdgeYes, prefixedChromium-based, so behavior tracks Chrome closely.
Safari (macOS and iOS)Yes, prefixedDifferent engine, different accuracy, stricter user-gesture requirements. Test separately — do not assume Chrome behavior.
FirefoxEffectively noThe interface has existed behind a flag for years and is not enabled for users. Plan for the fallback path.
Chrome on AndroidYes, prefixedBehavior varies by device and OS speech settings. Background tab handling is unreliable.
In-app browsersUnpredictableWebviews inside social and messaging apps frequently block microphone access outright.
Electron and embedded webviewsUsually noThe recognition service is a browser-vendor service, not part of Chromium. Shipping Chromium does not ship recognition.

That last row surprises people. webkitSpeechRecognition is not part of the open-source Chromium project's shipped functionality — it's wired to Google's cloud service through Chrome's proprietary keys. Build a desktop app on Electron, and the API may be present and simply never return a result.

The practical rule: feature-detect, always, and design the non-supporting path as a real path rather than an error message. If a third of your users are on Firefox or an in-app browser, "speech recognition is unavailable" is not a graceful degradation — it's a missing feature.

Performance considerations and limitations

Here's where I'll be blunt, because the gap between "this works in my demo" and "this works for users" is wider for the Web Speech API than for almost any other browser API.

You cannot choose or tune the model. There is no vocabulary hinting, no domain adaptation, no way to tell the recognizer that your users say "Levofloxacin" or "Kubernetes" or the name of your product. Whatever the browser vendor's general-purpose model does with unusual words is what you get, permanently. For consumer speech about everyday topics it's decent. For anything with a specialized vocabulary — medical, legal, financial, industrial, or just your own SKUs — accuracy falls off a cliff and you have no lever to pull.

Accuracy is not yours and it changes without warning. The engine behind webkitSpeechRecognition is updated by the browser vendor on their schedule. Your transcription quality can improve or regress on a Tuesday because Chrome shipped an update. There's no version pinning, no changelog you can subscribe to, and no way to reproduce a bug report from three months ago.

Behavior differs materially between browsers. Not just support — behavior. Session timeouts, silence handling, how interim results settle, punctuation, capitalization, and the actual words returned all differ between Chrome and Safari for identical audio. You will write branching code, and you'll discover the branches in production.

There's no speaker separation. One transcript stream, no indication of who's talking. Any use case involving two or more people in a room — meetings, interviews, calls, classrooms — is out of scope. If you need to know who said what, you need speaker diarization, which the Web Speech API does not have and will not get.

Microphone-only, no arbitrary audio. The API listens to the device's default input. You cannot feed it a MediaStream from a WebRTC call, a system audio capture, an <audio> element, or a file. That rules out the entire category of "transcribe this call" and "transcribe this recording" features.

Noise, distance, and accents. Because the engine is tuned for a phone or laptop mic held close to a cooperative speaker in a quiet room, everything else degrades. Conference rooms, drive-thrus, call center floors, kiosks, and strong accents are exactly where a general-purpose consumer model is weakest, and you have no far-field setting to reach for.

Privacy is not under your control. Your users' microphone audio leaves their device and goes to a third party's servers through a path you don't own and can't audit. For a consumer toy that's fine. For anything touching regulated data, an enterprise security review, or a data residency commitment, "the browser sends it somewhere" is not an answer you can give.

Silent failure modes are the worst part. The API rarely announces that it isn't working. It returns an empty transcript, or it stops firing events, or it never fires onresult at all. There is nothing to log and no status code to check.

The honest summary: the Web Speech API is excellent for a voice search field, a "fill this form by speaking" affordance, an accessibility shortcut, or a prototype you're showing to a stakeholder on your own laptop. It is not a foundation for a product where transcription quality is part of what people are paying for.

Moving to a streaming speech-to-text API

When the browser API runs out, the shape of the replacement is straightforward: capture audio in the browser yourself, stream it over a WebSocket to a speech-to-text API you control, and get transcripts back over the same connection.

You get back everything the browser API took away — model choice, keyterm biasing, speaker labels, the ability to transcribe any audio source rather than just the default mic, consistent behavior across every browser including Firefox, and a vendor relationship where accuracy regressions are somebody's job.

AssemblyAI's streaming speech-to-text endpoint is a single WebSocket at wss://streaming.assemblyai.com/v3/ws. The current flagship streaming model is Universal-3.5 Pro Realtime, specified as speech_model=universal-3-5-pro, at $0.45/hr base billed on session duration.

On Pipecat's open speech-to-text benchmark — real agent conversations rather than read-aloud audiobook audio — it looks like this:

Metric (lower is better)Universal-3.5 Pro RealtimeDeepgram FluxElevenLabs Scribe v2Google Chirp3
Word error rate6.99%15.58%9.76%9.04%
Entity error rate15.31%50.50%39.70%21.51%

Entity error rate is the row that predicts whether users trust the feature. Names, addresses, phone numbers, and account IDs are the words that carry the meaning of a sentence, and they're the words a general-purpose model gets wrong. More numbers across models and datasets live on the benchmarks page.

Step 1: mint a temporary token on your server

Never put an API key in client-side JavaScript. Generate a short-lived token server-side instead. Here's a complete Node server that both serves your page and issues tokens:

// server.js — run with: node server.js
import express from "express";

const app = express();
const API_KEY = process.env.ASSEMBLYAI_API_KEY;

app.use(express.static("public"));

app.get("/token", async (_req, res) => {
  const url = new URL("https://streaming.assemblyai.com/v3/token");
  url.searchParams.set("expires_in_seconds", "60");
  url.searchParams.set("max_session_duration_seconds", "600");

  const response = await fetch(url, { headers: { authorization: API_KEY } });

  if (!response.ok) {
    return res.status(response.status).json({ error: await response.text() });
  }

  res.json(await response.json());
});

app.listen(3000, () => console.log("http://localhost:3000"));

expires_in_seconds is the redemption window — how long the client has to open the WebSocket — not the session length. max_session_duration_seconds caps the session itself.

Step 2: capture and stream audio from the browser

The browser side captures microphone audio with getUserMedia, converts it to 16-bit PCM in an AudioWorklet, and sends it as binary frames.

One constraint shapes this whole file, and getting it wrong is the most common way this integration fails: the streaming API accepts between 50 ms and 1000 ms of audio per WebSocket message. An AudioWorkletProcessor is called with a fixed render quantum of 128 samples, which is always less than 50 ms — 8 ms at a 16 kHz context, 2.67 ms at 48 kHz. Post each quantum straight to the socket and the very first one comes back as:

{"type":"Error","error_code":3007,
 "error":"Input Duration Error: Input Duration Violation: 8.0 ms. Expected between 50 and 1000 ms"}

…and the session closes. So the worklet accumulates samples and only posts once it has a legal chunk. Save this as public/worklet.js:

// public/worklet.js — converts Float32 frames to 16-bit PCM and batches them
// to 100 ms, because the streaming API rejects messages shorter than 50 ms.
// process() is always handed 128 samples, which is never enough on its own.
const BATCH_MS = 100;

class PCMProcessor extends AudioWorkletProcessor {
  constructor() {
    super();
    // sampleRate is a global inside the AudioWorklet scope.
    this.batchSize = Math.round((sampleRate * BATCH_MS) / 1000);
    this.buffer = new Int16Array(this.batchSize);
    this.offset = 0;
  }

  process(inputs) {
    const input = inputs[0]?.[0];
    if (!input) return true;

    for (let i = 0; i < input.length; i++) {
      const sample = Math.max(-1, Math.min(1, input[i]));
      this.buffer[this.offset++] = sample < 0 ? sample * 0x8000 : sample * 0x7fff;

      if (this.offset === this.batchSize) {
        // Copy out, then transfer the copy so our buffer stays usable.
        const chunk = this.buffer.slice();
        this.port.postMessage(chunk.buffer, [chunk.buffer]);
        this.offset = 0;
      }
    }

    return true;
  }
}

registerProcessor("pcm-processor", PCMProcessor);

The batch boundary deliberately doesn't align with the render quantum — 100 ms at 16 kHz is 1600 samples, or 12.5 quanta — so the loop fills, flushes, and keeps going mid-quantum rather than counting callbacks.

And public/index.html:

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8" />
    <title>Streaming speech-to-text in JavaScript</title>
    <style>
      body { font-family: system-ui, sans-serif; max-width: 42rem; margin: 3rem auto; padding: 0 1rem; }
      #transcript { min-height: 8rem; border: 1px solid #ccc; padding: 1rem; margin-top: 1rem; }
      .partial { color: #888; }
    </style>
  </head>
  <body>
    <h1>Streaming speech-to-text</h1>
    <button id="start">Start</button>
    <button id="stop" disabled>Stop</button>
    <p id="status">Idle.</p>
    <div id="transcript"></div>

    <script type="module">
      const startBtn = document.getElementById("start");
      const stopBtn = document.getElementById("stop");
      const statusEl = document.getElementById("status");
      const transcriptEl = document.getElementById("transcript");

      let socket, audioContext, stream, workletNode;
      let finalText = "";

      async function start() {
        startBtn.disabled = true;

        // Start the microphone FIRST so we know the real sample rate. Browsers
        // are allowed to ignore the sampleRate hint, and sending the wrong
        // value produces a transcript that reads like static rather than an
        // error.
        statusEl.textContent = "Requesting microphone…";
        stream = await navigator.mediaDevices.getUserMedia({
          audio: {
            channelCount: 1,
            echoCancellation: true,
            noiseSuppression: true,
          },
        });

        audioContext = new AudioContext({ sampleRate: 16000 });
        await audioContext.audioWorklet.addModule("/worklet.js");

        statusEl.textContent = "Getting token…";
        const { token } = await fetch("/token").then((r) => r.json());

        const params = new URLSearchParams({
          sample_rate: String(audioContext.sampleRate),
          speech_model: "universal-3-5-pro",
          format_turns: "true",
          token,
        });

        socket = new WebSocket(
          `wss://streaming.assemblyai.com/v3/ws?${params.toString()}`
        );
        socket.binaryType = "arraybuffer";

        socket.onopen = () => {
          statusEl.textContent = "Listening…";
          stopBtn.disabled = false;
          startMicrophone();
        };

        socket.onmessage = (event) => {
          const data = JSON.parse(event.data);

          if (data.type === "Turn") {
            if (data.end_of_turn) {
              finalText += data.transcript + " ";
              transcriptEl.textContent = finalText;
            } else {
              transcriptEl.innerHTML =
                finalText + `<span class="partial">${data.transcript}</span>`;
            }
          }

          if (data.type === "Error") {
            statusEl.textContent = `API error ${data.error_code}: ${data.error}`;
          }

          if (data.type === "Termination") {
            statusEl.textContent = `Session ended after ${data.session_duration_seconds}s.`;
          }
        };

        socket.onerror = () => {
          statusEl.textContent = "Connection error.";
        };

        socket.onclose = () => {
          statusEl.textContent = "Disconnected.";
          startBtn.disabled = false;
          stopBtn.disabled = true;
        };
      }

      function startMicrophone() {
        const source = audioContext.createMediaStreamSource(stream);
        workletNode = new AudioWorkletNode(audioContext, "pcm-processor");

        workletNode.port.onmessage = ({ data }) => {
          if (socket && socket.readyState === WebSocket.OPEN) {
            socket.send(data);
          }
        };

        source.connect(workletNode);
        // Do NOT connect the worklet to destination — that echoes the mic back
        // through the speakers. A zero-gain sink keeps the graph pulling.
        const sink = audioContext.createGain();
        sink.gain.value = 0;
        workletNode.connect(sink).connect(audioContext.destination);
      }

      function stop() {
        stopBtn.disabled = true;

        if (socket && socket.readyState === WebSocket.OPEN) {
          // Terminate finalizes the open turn and stops the billing clock.
          socket.send(JSON.stringify({ type: "Terminate" }));
        }

        workletNode?.disconnect();
        stream?.getTracks().forEach((track) => track.stop());
        audioContext?.close();
      }

      startBtn.addEventListener("click", start);
      stopBtn.addEventListener("click", stop);
      window.addEventListener("beforeunload", stop);
    </script>
  </body>
</html>

Run it with ASSEMBLYAI_API_KEY=your-key node server.js and open http://localhost:3000. That's roughly 120 lines of JavaScript for browser speech recognition that works in Firefox, gives you a model you selected, and doesn't quietly restart itself every sixty seconds.

Three details in there are load-bearing:

  • The mic starts before the socket opens. sample_rate has to match the audio you actually send. Browsers may ignore the sampleRate hint on AudioContext, so the code reads audioContext.sampleRate back and sends that. Guess wrong and you get a fluent transcript of nothing in particular, with no error to tell you why.
  • The worklet batches to 100 ms. Covered above. This is the difference between a working demo and error 3007.
  • The worklet feeds a zero-gain sink, not the speakers. Connecting it straight to destination routes the microphone back out of the user's speakers.

One billing detail people get wrong: streaming is billed on how long the WebSocket stays open, not on how much audio you send. Sending the Terminate message on stop and on beforeunload is not tidiness — it's the difference between paying for a conversation and paying for an idle tab.

Stream Speech From Your Own Browser App

One WebSocket, temporary tokens for client-side auth, and per-second billing with no minimums. Get a key and run the code above in a few minutes.

Sign up free

How end-of-turn detection works

The Web Speech API decides a speaker has finished by watching for silence, and you can feel it. Pause to think mid-sentence and the session ends. Trail off softly and it hangs. Say "my email is" and then read it out slowly, and the recognizer cuts you in half.

Silence is a bad proxy for "done talking," because humans don't stop talking at predictable intervals. They pause inside a thought as often as after one.

Universal-3.5 Pro Realtime approaches this differently: rather than timing the gap alone, the model evaluates what has been said — whether the utterance is semantically complete — and combines that judgement with silence thresholds. An unfinished sentence buys the speaker more time; a complete one closes the turn sooner. That's why reading out an account number digit by digit doesn't get chopped in half the way a pure silence timer chops it.

You steer it with modes rather than hand-tuning thresholds:

ModeUse it formin_turn_silencemax_turn_silence
min_latencyConversational agents where a fast reply beats a perfect one128 ms640 ms
balanced (default)Most applications, including browser dictation and live captions128 ms1280 ms
max_accuracyNoisy rooms, far-field mics, kiosks, drive-thrus512 ms2560 ms

Those silence values are the mode defaults; vad_threshold defaults to 0.2. You can override any of them directly, but the modes exist because the combinations that work well together are not obvious. Note that turning on speaker_labels changes the picture — it shifts the defaults and disables continuous partials.

Each Turn message also carries an end_of_turn_confidence value, which is what you want if you're building a voice agent and would rather wait a moment on a borderline turn than talk over someone:

socket.onmessage = (event) => {
  const data = JSON.parse(event.data);
  if (data.type !== "Turn") return;

  if (data.end_of_turn && data.end_of_turn_confidence > 0.8) {
    handleCompletedUtterance(data.transcript); // confident: respond now
  } else if (data.end_of_turn) {
    scheduleWithGrace(data.transcript, 400); // unsure: give them a beat
  } else {
    renderPartial(data.transcript);
  }
};

Server-side, the Python SDK exposes the same stream through RealTimeTranscriber, which manages the connection and session termination for you:

import os

from assemblyai.streaming.v3 import (
    BeginEvent,
    RealTimeEvents,
    RealTimeParameters,
    RealTimeTranscriber,
    TerminationEvent,
    TurnEvent,
)


def on_begin(client: RealTimeTranscriber, event: BeginEvent) -> None:
    print(f"Session started: {event.id}")


def on_turn(client: RealTimeTranscriber, event: TurnEvent) -> None:
    if not event.transcript:
        return
    marker = "FINAL" if event.end_of_turn else "     "
    print(f"{marker} {event.transcript}")


def on_terminated(client: RealTimeTranscriber, event: TerminationEvent) -> None:
    print(f"Processed {event.audio_duration_seconds}s of audio")


client = RealTimeTranscriber(api_key=os.environ["ASSEMBLYAI_API_KEY"])
client.on(RealTimeEvents.Begin, on_begin)
client.on(RealTimeEvents.Turn, on_turn)
client.on(RealTimeEvents.Termination, on_terminated)

client.connect(RealTimeParameters(speech_model="universal-3-5-pro"))

# Feed it 16 kHz mono PCM chunks from any source: a mic, a WebRTC track,
# a telephony bridge, or a file. 3200 bytes is 100 ms at 16 kHz — the same
# 50-1000 ms rule applies here as in the browser.
with open("meeting.pcm", "rb") as audio:
    while chunk := audio.read(3200):
        client.stream(chunk)

client.disconnect(terminate=True)

Migrating from OpenAI's Realtime API without rewriting your agent

A common path to this page: you built a voice agent on OpenAI's Realtime API, it works, and transcription quality — usually on names, addresses, and account numbers — is the thing users complain about.

You don't have to rewrite the agent to fix that. The Realtime API bundles speech-to-text, the language model, and text-to-speech into one socket, but you can split the speech-to-text out and leave the rest alone. Your agent logic, your prompts, your tools, your voice all stay exactly where they are.

The move is mechanical:

  1. Stop sending audio to the Realtime socket. Keep the session for text-in, audio-out.
  2. Open a second WebSocket to wss://streaming.assemblyai.com/v3/ws and send your microphone audio there instead.
  3. On each finalized turn, push the transcript into your existing conversation as a text message rather than letting the model transcribe audio itself.
// Audio now goes to AssemblyAI; finalized text goes to your existing agent.
stt.onmessage = (event) => {
  const data = JSON.parse(event.data);
  if (data.type !== "Turn" || !data.end_of_turn || !data.transcript) return;

  agent.send(
    JSON.stringify({
      type: "conversation.item.create",
      item: {
        type: "message",
        role: "user",
        content: [{ type: "input_text", text: data.transcript }],
      },
    })
  );

  agent.send(JSON.stringify({ type: "response.create" }));
};

Everything downstream of that — tool calls, system prompt, response handling — is untouched. If you're using a framework, it's even less work: AssemblyAI ships drop-in plugins for both Pipecat and LiveKit, so swapping the speech-to-text service is a constructor change rather than a plumbing exercise.

One addition worth making while you're in there. Universal-3.5 Pro Realtime accepts an agent_context field: pass in the question your agent just asked, and the model resolves short replies against it. "Is it S as in Sierra?" gets transcribed correctly because the model knows a spelled-out name is what's expected. Across a benchmark of more than 10,000 voice agent audio files, passing agent context cut word error rate by 8.9% — rising to 16.4% when a context prompt is supplied alongside it, with fabrications down 27.0%.

There's more on where these stacks strain at scale in the production ceiling, and on evaluation criteria in choosing a speech-to-text API for voice agents.

The Dictation API: voice input that returns send-ready text

There's a category of browser voice feature the Web Speech API is nominally built for and structurally bad at: dictation. Not captioning a meeting — one person talking to their own app, writing a message, filling a field, drafting a note.

What makes dictation hard isn't recognition. It's that people don't speak in finished prose. They say "um, so, can you send me the — actually the Q3 numbers, before the meeting tomorrow." A verbatim transcript of that is accurate and unusable. What the user wants in the text box is "Can you send me the Q3 numbers before the meeting tomorrow?"

The Web Speech API gives you the first one. Getting to the second means bolting an LLM cleanup pass onto the end, which adds a round trip, a second vendor, and a new failure mode.

AssemblyAI's Dictation API does both in a single HTTP call. You send an utterance and get back two fields: text, the verbatim transcript, and llm_response, the cleaned-up version the user can send as-is — filler removed, self-corrections resolved to what the speaker actually landed on, punctuation and capitalization applied, and the names and terms they care about spelled the way they spell them. It runs on Universal-3.5 Pro across 32 languages and accepts up to 120 seconds of audio per call.

Worth being precise about what this is: the Dictation API is an API, not an app. There's no window, no global hotkey, no menu bar icon. It's the service layer you'd build a voice-input feature on — the thing you reach for when your product needs its own dictation, sitting inside your own UI, with your own vocabulary and your own output format.

import assemblyai as aai

aai.settings.api_key = "<YOUR_API_KEY>"

result = aai.DictationTranscriber().transcribe_live("clip.wav")

print("verbatim:", result.text)

# The rewrite is best effort on a 5-second deadline. On a timeout you still get
# HTTP 200, with llm_response None and llm_error set — so always fall back.
if result.llm_response:
    print("cleaned: ", result.llm_response)
else:
    print("cleaned: ", result.text, f"(rewrite skipped: {result.llm_error})")
verbatim: Um, so, can you send me the Q3 numbers before the meeting tomorrow? Thanks.
cleaned:  Can you send me the Q3 numbers before the meeting tomorrow? Thanks.

That fallback is not defensive padding. The cleanup runs on a five-second deadline and a timeout is reported as a successful request with llm_response: null and llm_error: "timeout" — so code that prints llm_response directly will happily show your users the word "None".

Cleanup runs by default with no configuration. When your app needs a different shape — a bulleted task list, a structured note — llm_instruction replaces the default cleanup task with whatever you describe in plain English. keyterms_prompt biases transcription toward names and jargon your users actually say.

From JavaScript, it's a multipart POST with the config part first:

async function dictate(audioBlob) {
  const form = new FormData();

  // `config` must come first — the server starts transcribing as audio arrives.
  form.append(
    "config",
    new Blob([JSON.stringify({ keyterms_prompt: ["Kubernetes", "Grafana"] })], {
      type: "application/json",
    })
  );
  form.append("audio", audioBlob, "clip.wav");

  const response = await fetch("/dictate", { method: "POST", body: form });
  const result = await response.json();

  // `llm_response` can be null if the rewrite failed; fall back to verbatim.
  return result.llm_response ?? result.text;
}

Proxy that through your own server so the API key stays server-side, the same way the token endpoint does above. Dictation has its own hostname, dictation.assemblyai.com, and accepts WAV or raw 16-bit PCM — the same PCM your AudioWorklet is already producing.

Web Speech API vs a streaming speech-to-text API

CapabilityWeb Speech APIStreaming speech-to-text API
Setup15 lines, no accountA token endpoint and an audio worklet
CostFreePer second of session time
Browser coverageChromium and Safari onlyAnything with WebSockets, including Firefox
Model controlNoneModel choice, keyterms, prompting, pinned versions
Speaker labelsNoYes, live, with revision
Audio sourcesDefault microphone onlyAny PCM stream: WebRTC, telephony, system audio, files
End-of-turn detectionSilence timersSemantic completeness plus tuned silence thresholds, with a confidence score
Data handlingBrowser vendor's serversYour vendor, with EU data residency available

The decision isn't really about features. It's about whether transcription quality is incidental to your product or part of what people are paying for. A voice search box where a misheard word costs the user one retry can live on the browser API forever. A notetaker, a call analytics tool, a clinical scribe, or a voice agent cannot, because in those products the transcript is the product, and you can't ship something whose accuracy you don't control and can't improve.

If you want the same pipeline for recorded rather than live audio, real-time transcription in Python covers the streaming side server-first, and the free speech-to-text APIs and open-source engines roundup covers what's available if you'd rather run something yourself. Current per-second rates for every model are on the pricing page, and the Voice Agent API bundles speech-to-text, the language model, and text-to-speech behind one socket if you'd rather not assemble the pipeline at all.

Where this leaves you

The most useful thing to internalize about the Web Speech API is that it was never really a speech recognition API. It's a permission slip — a standardized way to ask the browser to hand your users' voice to whichever recognizer the vendor picked. That design made it trivial to adopt and impossible to depend on, and thirteen years later both halves of that trade are still true.

Which is fine, as long as you know which side of the line your feature sits on. Build the fifteen-line version first. It costs you nothing, and it tells you very quickly whether voice input is something your users actually want. If they do, you'll know within a week — because the complaints you get will all be about the words, and the words are the one thing the browser API won't let you fix.

Get A Free Key And Swap The Engine

Keep your browser UI and replace what happens to the audio. Universal-3.5 Pro Realtime over one WebSocket, with temporary tokens built for client-side apps.

Sign up free

Frequently asked questions

What is the Web Speech API?

The Web Speech API is a browser standard that gives JavaScript access to speech recognition (SpeechRecognition) and text-to-speech (SpeechSynthesis). It's an interface rather than an engine — your code talks to the browser, and the browser hands the audio to its own recognition service, which is Google's in Chrome and Apple's in Safari. That means the accuracy, language support, and behavior you get are set by the browser vendor, not by you.

Which browsers support the Web Speech API?

SpeechRecognition works in Chrome, Edge, Safari, and Chrome on Android, in every case behind the webkitSpeechRecognition prefix. Firefox does not enable it for users, and Electron and other embedded webviews usually cannot use it because the recognition service is tied to the browser vendor's cloud rather than to Chromium itself. Always feature-detect with window.SpeechRecognition || window.webkitSpeechRecognition and design a real fallback path.

How do I use JavaScript speech recognition in the browser?

Create a recognizer with new (window.SpeechRecognition || window.webkitSpeechRecognition)(), set continuous and interimResults, handle the result event, and call .start(). Two details make or break it: iterate results from event.resultIndex rather than zero, because event.results is cumulative, and restart the recognizer in onend, because Chrome ends sessions on its own even when continuous is true. A complete working page is in this post.

Why does the Web Speech API stop listening on its own?

Chrome terminates recognition sessions periodically and after stretches of silence, regardless of the continuous setting, and you can't configure the timeout. The standard workaround is to call .start() again inside the onend handler, guarded by a flag so a deliberate stop doesn't restart it. If you need sessions that genuinely run for hours, a streaming speech-to-text API over a WebSocket is the reliable path.

Is the Web Speech API accurate enough for production?

It depends entirely on vocabulary and environment. For everyday consumer speech in a quiet room it's reasonable, but there's no way to bias it toward domain terms, product names, or proper nouns, no speaker separation, and no version pinning — accuracy can change when the browser updates. On Pipecat's open speech-to-text benchmark of real agent conversations, Universal-3.5 Pro Realtime records a 6.99% word error rate and a 15.31% entity error rate, and entity accuracy is what usually decides whether users trust a voice feature.

Why does my streaming WebSocket close with error 3007?

You're sending audio messages that are too short. The streaming API accepts between 50 ms and 1000 ms of audio per message, and an AudioWorkletProcessor is always handed exactly 128 samples — 8 ms at 16 kHz, less at higher rates — so posting each render quantum straight to the socket fails on the first one. Buffer inside the worklet until you have at least 50 ms (100 ms is a good default) and send that instead.