> ## Documentation Index
> Fetch the complete documentation index at: https://assemblyai.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Add context mid-dictation

> Change keyterms, the STT prompt or the rewrite after the user starts talking, by restarting the request with the audio so far.

The `config` part arrives before the audio, so a [live upload](https://assemblyai.com/docs/dictation/uploading-while-recording.md)
can't change its settings once the audio has started. Some context only arrives
after the user starts speaking: names pulled from the screen, a message that
comes in while they talk, the app they switch to. To use it, restart the
request:

1. **Keep a copy of every audio frame** you send, from the first one.
2. **When the new context is ready, open a new request** with the updated
   `config`: `stt_prompt`, `keyterms_prompt`, `llm_instruction` or `llm`.
3. **Send the copy first, then keep streaming.** The copy can go faster than
   real time, so the new request catches up with the speaker.
4. **Abort the old request.** Abort it rather than ending its audio: ending the
   audio asks for a transcript, and a finished request is billed. A request
   aborted before it finishes returns nothing and is not billed.

Restart once, when the context is ready, rather than on every change. Each
restart transcribes the audio so far again, which adds a little latency on a
long recording. The 120-second limit counts all the audio in the new request,
the copy included; see [Recordings over two minutes](https://assemblyai.com/docs/dictation/recordings-over-two-minutes.md).

## Example

Each example records 16 kHz mono 16-bit PCM from the default microphone,
starts with a plain config, and restarts with keyterms three seconds in, where
your own screen-context lookup would call it. The SDK examples use a live
session's `abort()`; the examples without an SDK drop the connection before the
body is complete. The JavaScript examples record with
[SoX](https://sourceforge.net/projects/sox/), so install it first
(`brew install sox` or `sudo apt-get install sox`).

> [tab] Python SDK
>
> ```python expandable theme={null}
> import threading
>
> import assemblyai as aai
> import sounddevice as sd
>
> aai.settings.api_key = "<YOUR_API_KEY>"
>
>
> class RestartableDictation:
>     """A live dictation whose config can change while the user is talking."""
>
>     def __init__(self, transcriber, config):
>         self._transcriber = transcriber
>         self._audio = bytearray()
>         self._lock = threading.Lock()
>         self._session = transcriber.open_live(config)
>
>     def write(self, chunk):
>         """Call from the microphone callback with each audio frame."""
>         with self._lock:
>             self._audio += chunk
>             self._session.write(chunk)
>
>     def update_config(self, config):
>         """Restart with a new config, replaying the audio captured so far."""
>         new_session = self._transcriber.open_live(config)
>         with self._lock:
>             old_session, self._session = self._session, new_session
>             new_session.write(bytes(self._audio))
>         old_session.abort()  # not close(): an aborted request is not billed
>
>     def result(self):
>         with self._lock:
>             session = self._session
>         return session.result()
>
>
> config = aai.DictationConfig(sample_rate=16000, channels=1, rewrite_mode="smart")
> with_context = aai.DictationConfig(
>     sample_rate=16000,
>     channels=1,
>     rewrite_mode="smart",
>     keyterms_prompt=["Priya Shah", "Q3 roadmap"],
> )
>
> with aai.DictationTranscriber() as transcriber:
>     dictation = RestartableDictation(transcriber, config)
>     stream = sd.RawInputStream(
>         samplerate=16000,
>         channels=1,
>         dtype="int16",
>         callback=lambda data, *_: dictation.write(bytes(data)),
>     )
>     with stream:
>         # Your screen-context lookup calls update_config when it's ready.
>         threading.Timer(3, dictation.update_config, args=[with_context]).start()
>         input("Dictating, press Enter to stop... ")
>
>     print(dictation.result().llm_response)
> ```

> [tab] JavaScript SDK
>
> ```javascript expandable theme={null}
> import { spawn } from "node:child_process";
> import { AssemblyAI } from "assemblyai";
>
> const client = new AssemblyAI({ apiKey: "<YOUR_API_KEY>" });
>
> // A live dictation whose config can change while the user is talking.
> function restartableDictation(config) {
>   const frames = [];
>   let session = client.dictation.openLive(config);
>
>   return {
>     // Call with each audio frame from the microphone.
>     write(chunk) {
>       frames.push(chunk);
>       session.write(chunk);
>     },
>     // Restart with a new config, replaying the audio captured so far.
>     async updateConfig(newConfig) {
>       const oldSession = session;
>       session = client.dictation.openLive(newConfig);
>       for (const frame of frames) session.write(frame);
>       await oldSession.abort(); // not close(): an aborted request is not billed
>     },
>     result() {
>       return session.result();
>     },
>   };
> }
>
> const config = { sample_rate: 16000, channels: 1, rewrite_mode: "smart" };
> const dictation = restartableDictation(config);
>
> // SoX writes 16 kHz, 16-bit mono PCM from the default microphone to stdout.
> const microphone = spawn("sox", [
>   "--default-device", "--no-show-progress",
>   "--rate", "16000", "--channels", "1",
>   "--encoding", "signed-integer", "--bits", "16", "--type", "raw", "-",
> ]);
> microphone.on("error", () => console.error("Install SoX to record audio."));
> microphone.stdout.on("data", (chunk) => dictation.write(chunk));
>
> // Your screen-context lookup calls updateConfig when it's ready.
> setTimeout(() => {
>   dictation.updateConfig({ ...config, keyterms_prompt: ["Priya Shah", "Q3 roadmap"] });
> }, 3000);
>
> console.log("Dictating, press Enter to stop...");
> process.stdin.once("data", () => {
>   microphone.kill();
>   process.stdin.pause();
> });
>
> await new Promise((resolve) => microphone.on("close", resolve));
> const result = await dictation.result();
> console.log(result.llm_response);
> ```

> [tab] Python
>
> ```python expandable theme={null}
> import json
> import queue
> import threading
>
> import requests
> import sounddevice as sd
>
> API_KEY = "<YOUR_API_KEY>"
> URL = "https://dictation.assemblyai.com/v1/transcribe/live"
> BOUNDARY = "----dictation-boundary"
>
>
> class Aborted(Exception):
>     """Raised inside the body so the connection drops before the body ends."""
>
>
> class LiveRequest:
>     """One upload to the live endpoint, fed frame by frame."""
>
>     def __init__(self, config):
>         self._frames = queue.Queue()
>         self._aborted = False
>         self._outcome = {}
>         self._thread = threading.Thread(target=self._send, args=(config,))
>         self._thread.start()
>
>     def _body(self, config):
>         yield (
>             f"--{BOUNDARY}\r\n"
>             'Content-Disposition: form-data; name="config"\r\n'
>             "Content-Type: application/json\r\n\r\n"
>             f"{json.dumps(config)}\r\n"
>             f"--{BOUNDARY}\r\n"
>             'Content-Disposition: form-data; name="audio"; filename="audio"\r\n'
>             "Content-Type: audio/pcm\r\n\r\n"
>         ).encode()
>         while (frame := self._frames.get()) is not None:
>             yield frame
>         if self._aborted:
>             raise Aborted()
>         yield f"\r\n--{BOUNDARY}--\r\n".encode()
>
>     def _send(self, config):
>         try:
>             self._outcome["response"] = requests.post(
>                 URL,
>                 headers={
>                     "Authorization": API_KEY,
>                     "Content-Type": f"multipart/form-data; boundary={BOUNDARY}",
>                 },
>                 data=self._body(config),
>                 timeout=90,
>             )
>         except Exception as error:
>             self._outcome["error"] = error
>
>     def write(self, frame):
>         self._frames.put(frame)
>
>     def abort(self):
>         """Drop the request without a transcript. An aborted request is not billed."""
>         self._aborted = True
>         self._frames.put(None)
>         self._thread.join()
>
>     def result(self):
>         """End the audio and wait for the transcript."""
>         self._frames.put(None)
>         self._thread.join()
>         if "error" in self._outcome:
>             raise self._outcome["error"]
>         response = self._outcome["response"]
>         response.raise_for_status()
>         return response.json()
>
>
> class RestartableDictation:
>     """A live dictation whose config can change while the user is talking."""
>
>     def __init__(self, config):
>         self._audio = bytearray()
>         self._lock = threading.Lock()
>         self._request = LiveRequest(config)
>
>     def write(self, frame):
>         """Call from the microphone callback with each audio frame."""
>         with self._lock:
>             self._audio += frame
>             self._request.write(frame)
>
>     def update_config(self, config):
>         """Restart with a new config, replaying the audio captured so far."""
>         new_request = LiveRequest(config)
>         with self._lock:
>             old_request, self._request = self._request, new_request
>             new_request.write(bytes(self._audio))
>         old_request.abort()
>
>     def result(self):
>         with self._lock:
>             request = self._request
>         return request.result()
>
>
> config = {"sample_rate": 16000, "channels": 1, "rewrite_mode": "smart"}
> with_context = {**config, "keyterms_prompt": ["Priya Shah", "Q3 roadmap"]}
>
> dictation = RestartableDictation(config)
> stream = sd.RawInputStream(
>     samplerate=16000,
>     channels=1,
>     dtype="int16",
>     callback=lambda data, *_: dictation.write(bytes(data)),
> )
> with stream:
>     # Your screen-context lookup calls update_config when it's ready.
>     threading.Timer(3, dictation.update_config, args=[with_context]).start()
>     input("Dictating, press Enter to stop... ")
>
> print(dictation.result()["llm_response"])
> ```

> [tab] JavaScript
>
> ```javascript expandable theme={null}
> import { spawn } from "node:child_process";
>
> const API_KEY = "<YOUR_API_KEY>";
> const URL = "https://dictation.assemblyai.com/v1/transcribe/live";
> const BOUNDARY = "----dictation-boundary";
> const encoder = new TextEncoder();
>
> // One upload to the live endpoint, fed frame by frame. Needs Node.js 18+.
> function liveRequest(config) {
>   let body;
>   const stream = new ReadableStream({ start: (controller) => (body = controller) });
>   body.enqueue(encoder.encode(
>     `--${BOUNDARY}\r\n` +
>     'Content-Disposition: form-data; name="config"\r\n' +
>     "Content-Type: application/json\r\n\r\n" +
>     `${JSON.stringify(config)}\r\n` +
>     `--${BOUNDARY}\r\n` +
>     'Content-Disposition: form-data; name="audio"; filename="audio"\r\n' +
>     "Content-Type: audio/pcm\r\n\r\n",
>   ));
>   const aborter = new AbortController();
>   const response = fetch(URL, {
>     method: "POST",
>     headers: {
>       Authorization: API_KEY,
>       "Content-Type": `multipart/form-data; boundary=${BOUNDARY}`,
>     },
>     body: stream,
>     duplex: "half",
>     signal: aborter.signal,
>   });
>   response.catch(() => {}); // an aborted request rejects; nothing to keep
>
>   return {
>     write(frame) {
>       body.enqueue(frame);
>     },
>     // Drop the request without a transcript. An aborted request is not billed.
>     abort() {
>       aborter.abort();
>     },
>     // End the audio and wait for the transcript.
>     async result() {
>       body.enqueue(encoder.encode(`\r\n--${BOUNDARY}--\r\n`));
>       body.close();
>       const res = await response;
>       if (!res.ok) throw new Error(`${res.status} ${await res.text()}`);
>       return res.json();
>     },
>   };
> }
>
> // A live dictation whose config can change while the user is talking.
> function restartableDictation(config) {
>   const frames = [];
>   let request = liveRequest(config);
>
>   return {
>     write(frame) {
>       frames.push(frame);
>       request.write(frame);
>     },
>     updateConfig(newConfig) {
>       const oldRequest = request;
>       request = liveRequest(newConfig);
>       for (const frame of frames) request.write(frame);
>       oldRequest.abort();
>     },
>     result() {
>       return request.result();
>     },
>   };
> }
>
> const config = { sample_rate: 16000, channels: 1, rewrite_mode: "smart" };
> const dictation = restartableDictation(config);
>
> // SoX writes 16 kHz, 16-bit mono PCM from the default microphone to stdout.
> const microphone = spawn("sox", [
>   "--default-device", "--no-show-progress",
>   "--rate", "16000", "--channels", "1",
>   "--encoding", "signed-integer", "--bits", "16", "--type", "raw", "-",
> ]);
> microphone.on("error", () => console.error("Install SoX to record audio."));
> microphone.stdout.on("data", (chunk) => dictation.write(new Uint8Array(chunk)));
>
> // Your screen-context lookup calls updateConfig when it's ready.
> setTimeout(() => {
>   dictation.updateConfig({ ...config, keyterms_prompt: ["Priya Shah", "Q3 roadmap"] });
> }, 3000);
>
> console.log("Dictating, press Enter to stop...");
> process.stdin.once("data", () => {
>   microphone.kill();
>   process.stdin.pause();
> });
>
> await new Promise((resolve) => microphone.on("close", resolve));
> const result = await dictation.result();
> console.log(result.llm_response);
> ```

## Related pages

* [Achieve lowest latency](https://assemblyai.com/docs/dictation/uploading-while-recording.md) — the live upload this restarts
* [Prompting and keyterms](https://assemblyai.com/docs/dictation/prompting-and-keyterms.md) — what `stt_prompt` and `keyterms_prompt` do
* [Recordings over two minutes](https://assemblyai.com/docs/dictation/recordings-over-two-minutes.md) — the fallback past 120 seconds
