Skip to main content


US & EU

Overview

Replace generic “Speaker A” and “Speaker B” labels with real names or roles, no voice enrollment needed. Speaker Identification uses conversation content to infer who’s speaking and applies the identifiers you provide. It can infer roles from context clues, and can associate names when the names are present within the transcript. Example transformation: Before:
After (by name):
After (by role):
Speaker Identification requires Speaker Diarization. You must set speaker_labels: true in your transcription request.
To reliably identify speakers, your audio should contain clear, distinguishable voices and sufficient spoken audio from each speaker. The accuracy of Speaker Diarization depends on the quality of the audio and the distinctiveness of each speaker’s voice, which will have a downstream effect on the quality of Speaker Identification.

Choosing how to identify speakers

You can identify speakers by name or by role:
  • Know the speakers’ names? Use speaker_type: "name" with the names in speakers. Click here to learn more.
  • Know their roles but not names? Use speaker_type: "role" with roles like "Interviewer" or "Agent" in speakers. Click here to learn more.
  • Need better accuracy? Use speakers with description fields that provide context about what each speaker typically discusses. Click here to learn more.

How to use Speaker Identification

Include the speech_understanding parameter in your transcription request to identify speakers as part of transcription.

Identify by name

To identify speakers by name, use speaker_type: "name" with a list of speaker names in speakers. This is the most common approach when you know who is speaking in the audio.

Identify by role

To identify speakers by role instead of name, use speaker_type: "role" with role labels in speakers. This is useful for customer service calls, interviews, or any scenario where you know the roles but not the names.

Common role combinations

  • [{"role": "Agent"}, {"role": "Customer"}] - Customer service calls
  • [{"role": "AI Assistant"}, {"role": "User"}] - AI chatbot interactions
  • [{"role": "Support"}, {"role": "Customer"}] - Technical support calls
  • [{"role": "Interviewer"}, {"role": "Interviewee"}] - Interview recordings
  • [{"role": "Host"}, {"role": "Guest"}] - Podcast or show recordings
  • [{"role": "Moderator"}, {"role": "Panelist"}] - Panel discussions

Controlling Effort

Sometimes when you are using the speaker identification the results are not quite as good high quality as you want. To assist with this there is a configuration to increase the quality of the results at a higher cost. Speaker Identification has two effort modes currently: low and medium. The default is low.

Example

When to use low/default effort

The low/default effort works best for lower complexity situations. Some examples:
  • Transcripts less than 10m long
  • Transcripts where there are clearly segmented speakers
    • doctor/patient meeting
    • podcasts
    • customer support calls
    • note: audio size is less restrictive here
  • Names and/or roles are well demonstrated in the transcript
    • names clearly spoken
    • immediately clear who has which role
      • example: “Hi thank you for calling insert_company_name, how can I help you” clearly indicates a customer support role

When to use medium effort

The medium effort works best for higher complexity transcripts, where information is less clear and/or speakers are less defined. Some example:
  • Meeting call in a conferance room
  • Arguments and/or elevated conversations where individuals interrupt each other
  • Transcripts where one or more names are not clearly stated
    • note: names cannot be synthesized from nothing, we cannot extract names if none are present in the transcript
  • Transcripts where the roles are nuanced or not clear

Apply to an existing transcript

If you already have a completed transcript, you can add Speaker Identification in a separate request to the Speech Understanding API. This is useful when you want to re-identify speakers with different parameters, or when your workflow separates transcription from post-processing. First, transcribe your audio with speaker_labels: true. Once transcription is complete, send the transcript_id along with your speaker identification configuration to the Speech Understanding API.
The example above identifies speakers by name. To identify by role, keep the same two-step flow and set speaker_type: "role" with role labels in speakers (see Identify by role). The speakers metadata approach works with this flow too.

Adding speaker metadata

The speakers parameter lets you provide additional metadata about each speaker to help the model identify speakers based on conversational context. This is particularly useful when:
  • Speakers have similar voices but distinct roles or topics
  • You want to provide contextual clues about what each speaker typically discusses
  • You need more precise identification in complex multi-speaker scenarios
Each speaker object must include either a name or role (depending on speaker_type). Beyond that, you can add any additional properties you want. The name and role fields are reserved as strings, but all other properties are flexible and can be any structure.
Examples in this section are shown in Python for brevity. The same speaker_identification configuration works in any language.
At its simplest, you can provide a description alongside each speaker’s name or role:
For even more fine-tuned identification, you can include any additional custom properties on each speaker object, such as company, title, department, or any other fields that help describe the speaker:
You can use the same custom properties with role-based identification by replacing name with role in each speaker object.

API reference

Request

In a transcription request

Include the speech_understanding parameter directly in your transcription request (shown here with name-based identification):

On an existing transcript

Transcribe with speaker_labels: true, then send the completed transcript_id to the Speech Understanding API:

Request parameters

The following parameters are nested under speech_understanding.request.speaker_identification:

Response

The Speaker Identification API returns a modified version of your transcript with updated speaker labels in the utterances key.

Response fields

With Speaker Identification, the speaker field in utterances and words contains the identified name or role (e.g., "Michel Martin" or "Agent") instead of generic labels like "A", "B", "C". All other fields (text, start, end, confidence, words) remain unchanged from the standard transcription response.