Code Switching - AssemblyAI

Transcribe audio containing multiple languages with code switching detection. This feature enables accurate transcription of conversations where speakers naturally switch between languages during conversations.

Universal-3.5 Pro (Recommended)

Universal-3.5 Pro natively handles code switching across 18 languages. Set language_detection as True and the model follows speakers as they shift mid-sentence between languages like English and Spanish, French, Hindi or Mandarin, preserving exactly what was said without translating everything into a single language. See Universal-3.5 Pro code switching in action.

English <> French

I said something like, j'ai dit à mes étudiants que it's time to really pay attention to what the idea of code-switching is.

English <> Hindi

मेरा रुकने का तो बहुत मन है, but I have an exam to give tomorrow.

English <> Mandarin

But this sentence, 我父母不工作了, you can see the 了 at the end of the sentence indicates that the situation now, my parents don't work, is different from what it was before.

Quickstart

import assemblyai as aai

aai.settings.api_key = "<YOUR_API_KEY>"

# audio_file = "./local-file.mp3"
audio_file = "https://assembly.ai/code-switching-3"

config = aai.TranscriptionConfig(
    language_detection=True,
)

transcript = aai.Transcriber(config=config).transcribe(audio_file)

if transcript.status == "error":
  raise RuntimeError(f"Transcription failed: {transcript.error}")

print(transcript.text)

import requests
import time

base_url = "https://api.assemblyai.com"
headers = {"authorization": "<YOUR_API_KEY>"}

data = {
    "audio_url": "https://assembly.ai/code-switching-3",
    "language_detection": True,
}

response = requests.post(base_url + "/v2/transcript", headers=headers, json=data)

if response.status_code != 200:
    print(f"Error: {response.status_code}, Response: {response.text}")
    response.raise_for_status()

transcript_response = response.json()
transcript_id = transcript_response["id"]
polling_endpoint = f"{base_url}/v2/transcript/{transcript_id}"

while True:
    transcript = requests.get(polling_endpoint, headers=headers).json()
    if transcript["status"] == "completed":
        print(transcript["text"])
        break
    elif transcript["status"] == "error":
        raise RuntimeError(f"Transcription failed: {transcript['error']}")
    else:
        time.sleep(3)

import { AssemblyAI } from "assemblyai";

const client = new AssemblyAI({
  apiKey: "<YOUR_API_KEY>",
});

// You can use a local filepath:
// const audioFile = "./local-file.mp3";

// Or use a publicly-accessible URL:
const audioFile = "https://assembly.ai/code-switching-3";

const params = {
  audio: audioFile,
  language_detection: true,
};

const run = async () => {
  const transcript = await client.transcripts.transcribe(params);

  if (transcript.status === "error") {
    console.error(`Transcription failed: ${transcript.error}`);
    process.exit(1);
  }

  console.log(`\nFull Transcript:\n\n${transcript.text}\n`);
};

run();

const baseUrl = "https://api.assemblyai.com";
const headers = {
  authorization: "<YOUR_API_KEY>",
};

const data = {
  audio_url: "https://assembly.ai/code-switching-3",
  language_detection: true,
};

const url = `${baseUrl}/v2/transcript`;
let res = await fetch(url, {
  method: "POST",
  headers: { ...headers, "Content-Type": "application/json" },
  body: JSON.stringify(data),
});
if (!res.ok) throw new Error(`Error: ${res.status}`);
const response = await res.json();

const transcriptId = response.id;
const pollingEndpoint = `${baseUrl}/v2/transcript/${transcriptId}`;

while (true) {
  res = await fetch(pollingEndpoint, { headers });
  if (!res.ok) throw new Error(`Error: ${res.status}`);
  const transcriptionResult = await res.json();

  if (transcriptionResult.status === "completed") {
    console.log(transcriptionResult.text);
    break;
  } else if (transcriptionResult.status === "error") {
    throw new Error(`Transcription failed: ${transcriptionResult.error}`);
  } else {
    await new Promise((resolve) => setTimeout(resolve, 3000));
  }
}

Universal-2

While Universal-2 supports code switching, we recommend upgrading to Universal-3.5 Pro for best results. To enable code switching on Universal-2, set speech_models to universal-2, language_detection to true, and code_switching to true inside the language_detection_options parameter.

Quickstart

import assemblyai as aai

aai.settings.api_key = "<YOUR_API_KEY>"

audio_file = "./bilingual-audio.mp3"
# audio_file = "https://assembly.ai/wildfires.mp3"

config = aai.TranscriptionConfig(
    speech_models=["universal-2"],
    language_detection=True,
    language_detection_options=aai.LanguageDetectionOptions(
      code_switching=True
    )
)

transcript = aai.Transcriber(config=config).transcribe(audio_file)

if transcript.status == "error":
  raise RuntimeError(f"Transcription failed: {transcript.error}")

print(transcript.text)

import requests
import time

base_url = "https://api.assemblyai.com"

headers = {
    "authorization": "<YOUR_API_KEY>"
}

with open("./bilingual-audio.mp3", "rb") as f:
  response = requests.post(base_url + "/v2/upload",
                          headers=headers,
                          data=f)

upload_url = response.json()["upload_url"]

data = {
    "audio_url": upload_url,
    "speech_models": ["universal-2"],
    "language_detection": True,
    "language_detection_options": {
        "code_switching": True
    },
}

url = base_url + "/v2/transcript"
response = requests.post(url, json=data, headers=headers)

transcript_id = response.json()['id']
polling_endpoint = base_url + "/v2/transcript/" + transcript_id

while True:
  transcription_result = requests.get(polling_endpoint, headers=headers).json()

  if transcription_result['status'] == 'completed':
    print(f"Transcript: {transcription_result['text']}")
    break

  elif transcription_result['status'] == 'error':
    raise RuntimeError(f"Transcription failed: {transcription_result['error']}")

  else:
    time.sleep(3)

import { AssemblyAI } from "assemblyai";

const client = new AssemblyAI({
  apiKey: "<YOUR_API_KEY>",
});

// You can use a local filepath:
const audioFile = "./bilingual-audio.mp3";

// Or use a publicly-accessible URL:
// const audioFile = "<AUDIO_URL>";

const params = {
  audio: audioFile,
  speech_models: ["universal-2"],
  language_detection: true,
  language_detection_options: {
    code_switching: true,
  },
};

const run = async () => {
  const transcript = await client.transcripts.transcribe(params);

  if (transcript.status === "error") {
    console.error(`Transcription failed: ${transcript.error}`);
    process.exit(1);
  }

  console.log(`\nFull Transcript:\n\n${transcript.text}\n`);
};

run();

import fs from "fs-extra";

const baseUrl = "https://api.assemblyai.com";

const headers = {
  authorization: "<YOUR_API_KEY>",
};

const path = "./bilingual-audio.mp3";
const audioData = await fs.readFile(path);
let res = await fetch(`${baseUrl}/v2/upload`, {
  method: "POST",
  headers,
  body: audioData,
});
if (!res.ok) throw new Error(`Error: ${res.status}`);
const uploadResponse = await res.json();
const uploadUrl = uploadResponse.upload_url;

const data = {
  audio_url: uploadUrl,
  speech_models: ["universal-2"],
  language_detection: true,
  language_detection_options: {
    code_switching: true,
  },
};

const url = `${baseUrl}/v2/transcript`;
res = await fetch(url, {
  method: "POST",
  headers: { ...headers, "Content-Type": "application/json" },
  body: JSON.stringify(data),
});
if (!res.ok) throw new Error(`Error: ${res.status}`);
const response = await res.json();

const transcriptId = response.id;
const pollingEndpoint = `${baseUrl}/v2/transcript/${transcriptId}`;

while (true) {
  res = await fetch(pollingEndpoint, { headers });
  if (!res.ok) throw new Error(`Error: ${res.status}`);
  const transcriptionResult = await res.json();

  if (transcriptionResult.status === "completed") {
    console.log(transcriptionResult.text);
    break;
  } else if (transcriptionResult.status === "error") {
    throw new Error(`Transcription failed: ${transcriptionResult.error}`);
  } else {
    await new Promise((resolve) => setTimeout(resolve, 3000));
  }
}

Example API Response

When enabling code switching with automatic language detection, the two detected language codes with the highest confidence and their confidence will be included in the transcript JSON.

"language_detection_results": {
      "code_switching_languages": [
           {"language": "en", "confidence": 0.8},
           {"language": "es", "confidence": 0.7}
     ]
}

Manually Setting Language Codes

To manually set the language codes, you can use the language_codes parameter. A max of two language codes can be set and one code must be "en". For example, if your file contains both English and Spanish, it would be "language_codes": ["en", "es"].

import assemblyai as aai

aai.settings.api_key = "<YOUR_API_KEY>"

audio_file = "./bilingual-audio.mp3"
# audio_file = "https://assembly.ai/wildfires.mp3"

config = aai.TranscriptionConfig(
  speech_models=["universal-2"],
  language_codes=["en", "es"]  # English-Spanish code switching
)

transcript = aai.Transcriber(config=config).transcribe(audio_file)

if transcript.status == "error":
  raise RuntimeError(f"Transcription failed: {transcript.error}")

print(transcript.text)

import requests
import time

base_url = "https://api.assemblyai.com"

headers = {
    "authorization": "<YOUR_API_KEY>"
}

with open("./bilingual-audio.mp3", "rb") as f:
  response = requests.post(base_url + "/v2/upload",
                          headers=headers,
                          data=f)

upload_url = response.json()["upload_url"]

data = {
    "audio_url": upload_url,
    "speech_models": ["universal-2"],
    "language_codes": ["en", "es"]  # English-Spanish code switching
}

url = base_url + "/v2/transcript"
response = requests.post(url, json=data, headers=headers)

transcript_id = response.json()['id']
polling_endpoint = base_url + "/v2/transcript/" + transcript_id

while True:
  transcription_result = requests.get(polling_endpoint, headers=headers).json()

  if transcription_result['status'] == 'completed':
    print(f"Transcript: {transcription_result['text']}")
    break

  elif transcription_result['status'] == 'error':
    raise RuntimeError(f"Transcription failed: {transcription_result['error']}")

  else:
    time.sleep(3)

import { AssemblyAI } from "assemblyai";

const client = new AssemblyAI({
  apiKey: "<YOUR_API_KEY>",
});

// You can use a local filepath:
const audioFile = "./bilingual-audio.mp3";

// Or use a publicly-accessible URL:
// const audioFile = "<AUDIO_URL>";

const params = {
  audio: audioFile,
  speech_models: ["universal-2"],
  language_codes: ["en", "es"], // English-Spanish code switching
};

const run = async () => {
  const transcript = await client.transcripts.transcribe(params);

  if (transcript.status === "error") {
    console.error(`Transcription failed: ${transcript.error}`);
    process.exit(1);
  }

  console.log(`\nFull Transcript:\n\n${transcript.text}\n`);
};

run();

import fs from "fs-extra";

const baseUrl = "https://api.assemblyai.com";

const headers = {
  authorization: "<YOUR_API_KEY>",
};

const path = "./bilingual-audio.mp3";
const audioData = await fs.readFile(path);
let res = await fetch(`${baseUrl}/v2/upload`, {
  method: "POST",
  headers,
  body: audioData,
});
if (!res.ok) throw new Error(`Error: ${res.status}`);
const uploadResponse = await res.json();
const uploadUrl = uploadResponse.upload_url;

const data = {
  audio_url: uploadUrl,
  speech_models: ["universal-2"],
  language_codes: ["en", "es"], // English-Spanish code switching
};

const url = `${baseUrl}/v2/transcript`;
res = await fetch(url, {
  method: "POST",
  headers: { ...headers, "Content-Type": "application/json" },
  body: JSON.stringify(data),
});
if (!res.ok) throw new Error(`Error: ${res.status}`);
const response = await res.json();

const transcriptId = response.id;
const pollingEndpoint = `${baseUrl}/v2/transcript/${transcriptId}`;

while (true) {
  res = await fetch(pollingEndpoint, { headers });
  if (!res.ok) throw new Error(`Error: ${res.status}`);
  const transcriptionResult = await res.json();

  if (transcriptionResult.status === "completed") {
    console.log(transcriptionResult.text);
    break;
  } else if (transcriptionResult.status === "error") {
    throw new Error(`Transcription failed: ${transcriptionResult.error}`);
  } else {
    await new Promise((resolve) => setTimeout(resolve, 3000));
  }
}

Code Switching Confidence Threshold

The code_switching_confidence_threshold parameter controls how the model routes transcription when multiple languages are detected. When code switching is enabled, the model detects up to two languages per audio file and assigns each a confidence score.

Code Switching Routing BehaviorThis parameter controls routing, not rejection. Audio is always transcribed, even if confidence scores do not meet this threshold. To return an error instead of a low-confidence transcription, you can use language_confidence_threshold alongside this parameter.

The threshold determines which language is used for transcription using the following logic:

If the non-English language’s confidence score meets or exceeds the threshold, the audio is routed to that non-English language model.
If the non-English language’s confidence score falls below the threshold, the audio is routed to whichever language has the highest overall confidence, which may be English or non-English.
If both detected languages are non-English, the audio is always routed to whichever has the higher confidence score, regardless of the threshold.

Code Switching DefaultBy default, the code_switching_confidence_threshold parameter is set to 0.3. If you would like to disable this, make sure to set this parameter to 0. Setting code_switching_confidence_threshold to 0 means the non-English language is always used for routing, even with very low confidence.

import assemblyai as aai

aai.settings.api_key = "<YOUR_API_KEY>"

audio_file = "./bilingual-audio.mp3"
# audio_file = "https://assembly.ai/wildfires.mp3"

config = aai.TranscriptionConfig(
    speech_models=["universal-2"],
    language_detection=True,
    language_detection_options=aai.LanguageDetectionOptions(
      code_switching=True,
      code_switching_confidence_threshold=0.5 # Optional parameter - this is set to 0.3 by default
    )
)

transcript = aai.Transcriber(config=config).transcribe(audio_file)

if transcript.status == "error":
  raise RuntimeError(f"Transcription failed: {transcript.error}")

print(transcript.text)

import requests
import time

base_url = "https://api.assemblyai.com"

headers = {
    "authorization": "<YOUR_API_KEY>"
}

with open("./bilingual-audio.mp3", "rb") as f:
  response = requests.post(base_url + "/v2/upload",
                          headers=headers,
                          data=f)

upload_url = response.json()["upload_url"]

data = {
    "audio_url": upload_url,
    "speech_models": ["universal-2"],
    "language_detection": True,
    "language_detection_options": {
        "code_switching": True,
        "code_switching_confidence_threshold": 0.5 # Optional parameter - this is set to 0.3 by default
    },
}

url = base_url + "/v2/transcript"
response = requests.post(url, json=data, headers=headers)

transcript_id = response.json()['id']
polling_endpoint = base_url + "/v2/transcript/" + transcript_id

while True:
  transcription_result = requests.get(polling_endpoint, headers=headers).json()

  if transcription_result['status'] == 'completed':
    print(f"Transcript: {transcription_result['text']}")
    break

  elif transcription_result['status'] == 'error':
    raise RuntimeError(f"Transcription failed: {transcription_result['error']}")

  else:
    time.sleep(3)

import { AssemblyAI } from "assemblyai";

const client = new AssemblyAI({
  apiKey: "<YOUR_API_KEY>",
});

// You can use a local filepath:
const audioFile = "./bilingual-audio.mp3";

// Or use a publicly-accessible URL:
// const audioFile = "<AUDIO_URL>";

const params = {
  audio: audioFile,
  speech_models: ["universal-2"],
  language_detection: true,
  language_detection_options: {
    code_switching: true,
    code_switching_confidence_threshold: 0.5, // Optional parameter - this is set to 0.3 by default
  },
};

const run = async () => {
  const transcript = await client.transcripts.transcribe(params);

  if (transcript.status === "error") {
    console.error(`Transcription failed: ${transcript.error}`);
    process.exit(1);
  }

  console.log(`\nFull Transcript:\n\n${transcript.text}\n`);
};

run();

import fs from "fs-extra";

const baseUrl = "https://api.assemblyai.com";

const headers = {
  authorization: "<YOUR_API_KEY>",
};

const path = "./bilingual-audio.mp3";
const audioData = await fs.readFile(path);
let res = await fetch(`${baseUrl}/v2/upload`, {
  method: "POST",
  headers,
  body: audioData,
});
if (!res.ok) throw new Error(`Error: ${res.status}`);
const uploadResponse = await res.json();
const uploadUrl = uploadResponse.upload_url;

const data = {
  audio_url: uploadUrl,
  speech_models: ["universal-2"],
  language_detection: true,
  language_detection_options: {
    code_switching: true,
    code_switching_confidence_threshold: 0.5, // Optional parameter - this is set to 0.3 by default
  },
};

const url = `${baseUrl}/v2/transcript`;
res = await fetch(url, {
  method: "POST",
  headers: { ...headers, "Content-Type": "application/json" },
  body: JSON.stringify(data),
});
if (!res.ok) throw new Error(`Error: ${res.status}`);
const response = await res.json();

const transcriptId = response.id;
const pollingEndpoint = `${baseUrl}/v2/transcript/${transcriptId}`;

while (true) {
  res = await fetch(pollingEndpoint, { headers });
  if (!res.ok) throw new Error(`Error: ${res.status}`);
  const transcriptionResult = await res.json();

  if (transcriptionResult.status === "completed") {
    console.log(transcriptionResult.text);
    break;
  } else if (transcriptionResult.status === "error") {
    throw new Error(`Transcription failed: ${transcriptionResult.error}`);
  } else {
    await new Promise((resolve) => setTimeout(resolve, 3000));
  }
}

Support for 99 languages

Universal-3.5 Pro supports 18 languages, and for anything outside that set, the system automatically falls back to Universal-2, giving you coverage across 99 languages total without any extra configuration.

Model	Supported languages
`universal-3-5-pro`	Global English, Australian English, British English, US English, Spanish, French, German, Italian, Portuguese, Arabic, Danish, Dutch, Finnish, Hebrew, Hindi, Japanese, Mandarin, Norwegian, Swedish, Turkish, Vietnamese
`universal-2`	Global English, Australian English, British English, US English, Spanish, French, German, Italian, Portuguese, Dutch, Hindi, Japanese, Chinese, Finnish, Korean, Polish, Russian, Turkish, Ukrainian, Vietnamese, Afrikaans, Albanian, Amharic, Arabic, Armenian, Assamese, Azerbaijani, Bashkir, Basque, Belarusian, Bengali, Bosnian, Breton, Bulgarian, Burmese, Catalan, Croatian, Czech, Danish, Estonian, Faroese, Galician, Georgian, Greek, Gujarati, Haitian, Hausa, Hawaiian, Hebrew, Hungarian, Icelandic, Indonesian, Javanese, Kannada, Kazakh, Khmer, Lao, Latin, Latvian, Lingala, Lithuanian, Luxembourgish, Macedonian, Malagasy, Malay, Malayalam, Maltese, Maori, Marathi, Mongolian, Nepali, Norwegian, Norwegian Nynorsk, Occitan, Panjabi, Pashto, Persian, Romanian, Sanskrit, Serbian, Shona, Sindhi, Sinhala, Slovak, Slovenian, Somali, Sundanese, Swahili, Swedish, Swiss German, Tagalog, Tajik, Tamil, Tatar, Telugu, Thai, Tibetan, Turkmen, Urdu, Uzbek, Welsh

​Universal-3.5 Pro (Recommended)

English <> French

English <> Hindi

English <> Mandarin

​Quickstart

​Universal-2

​Quickstart

​Example API Response

​Manually Setting Language Codes

​Code Switching Confidence Threshold

​Support for 99 languages

Universal-3.5 Pro (Recommended)

Quickstart

Universal-2

Quickstart

Example API Response

Manually Setting Language Codes

Code Switching Confidence Threshold

Support for 99 languages