Speaker Diarization
The Speaker Diarization model lets you detect multiple speakers in an audio file and what each speaker said.
If you enable Speaker Diarization, the resulting transcript will return a list of utterances, where each utterance corresponds to an uninterrupted segment of speech from a single speaker.
Want to name your speakers?
Speaker Diarization assigns generic labels like “Speaker A” and “Speaker B” to distinguish between speakers. If you want to replace these labels with actual names or roles (e.g., “John Smith” or “Customer”), use Speaker Identification. Speaker Identification analyzes the conversation content to infer who is speaking and transforms your transcript from generic labels to meaningful identifiers.
Quickstart
Python SDK
Python
JavaScript SDK
JavaScript
To enable Speaker Diarization, set speaker_labels to True in the transcription config.
Set number of speakers expected
You can set the number of speakers expected in the audio file by setting the speakers_expected parameter.
Only use this parameter if you are certain about the number of speakers in the audio file.
Python SDK
Python
JavaScript SDK
JavaScript
Set a range of possible speakers
You can set a range of possible speakers in the audio file by setting the speaker_options parameter. By default, the model will return between 1 and 10 speakers.
This parameter is suitable for use cases where there is a known minimum/maximum number of speakers in the audio file that is outside the bounds of the default value of 1 to 10 speakers.
Setting max_speakers_expected too high may reduce diarization accuracy,
causing sentences from the same speaker to be split across multiple speaker
labels.
Python SDK
Python
JavaScript SDK
JavaScript
API reference
Request
Speakers Expected
Speaker Options
Response
The response also includes the request parameters used to generate the transcript.
Identify speakers by name
Speaker Diarization assigns generic labels like “Speaker A” and “Speaker B” to each speaker. If you want to replace these labels with actual names or roles, you can use Speaker Identification to transform your transcript.
Before Speaker Identification:
After Speaker Identification:
The following example shows how to transcribe audio with Speaker Diarization and then apply Speaker Identification to replace the generic speaker labels with actual names.
Python
JavaScript
For more details on Speaker Identification, including how to identify speakers by role and how to apply it to existing transcripts, see the Speaker Identification guide.
Frequently asked questions & troubleshooting
How can I improve the performance of the Speaker Diarization model?
To improve the performance of the Speaker Diarization model, it’s recommended to ensure that each speaker speaks for at least 30 seconds uninterrupted. Avoiding scenarios where a person only speaks a few short phrases like “Yeah”, “Right”, or “Sounds good” can also help. If possible, avoiding cross-talking can also improve performance.
How many speakers can the model handle?
By default, the upper limit on the number of speakers for Speaker Diarization
is 10. If you expect more than 10 speakers, you can use
speaker_options
to set a range of possible speakers. Please note, setting
max_speakers_expected too high may reduce diarization accuracy, causing
sentences from the same speaker to be split across multiple speaker labels.
How accurate is the Speaker Diarization model?
The accuracy of the Speaker Diarization model depends on several factors, including the quality of the audio, the number of speakers, and the length of the audio file. Ensuring that each speaker speaks for at least 30 seconds uninterrupted and avoiding scenarios where a person only speaks a few short phrases can improve accuracy. However, it’s important to note that the model isn’t perfect and may make mistakes, especially in more challenging scenarios.
Why is the speaker diarization not performing as expected?
The speaker diarization may be performing poorly if a speaker only speaks once or infrequently throughout the audio file. Additionally, if the speaker speaks in short or single-word utterances, the model may struggle to create separate clusters for each speaker. Lastly, if the speakers sound similar, there may be difficulties in accurately identifying and separating them. Background noise, cross-talk, or an echo may also cause issues.
When should I use speakers_expected and when should I use speaker_options to set a range of speakers?
When should I use speakers_expected and when should I use speaker_options to set a range of speakers?
speakers_expected should be used only when you are confident that your audio file contains exactly the number of speakers you specify. If this number is incorrect, the diarization process, being forced to find an incorrect number of speakers, may produce random splits of single-speaker segments or merge multiple speakers into one in order to return the specified number of speakers. There are various scenarios where the audio file may include unexpected speakers, such as playback of recorded audio during a conversation or background speech from other people. To account for such cases, it is generally recommended to use min_speakers_expected instead of speakers_expected and to set max_speakers_expected slightly higher (e.g., min_speakers_expected + 2) to allow some flexibility.