How to build a lecture capture system with speaker identification

This tutorial shows you how to build a lecture capture system that automatically works out who is speaking in a classroom recording. You'll write a Python application that records the room, sends the audio through speaker diarization, and produces speaker-attributed captions students can search. It handles the things real classrooms throw at you: HVAC noise, a professor who wanders away from the mic, and a Q&A section where six people talk in ninety seconds.

You'll build two paths. The async path uploads a finished recording and returns the most accurate speaker labels available — that's what you want for the version students watch later. The streaming path labels speakers live over a WebSocket, for captions on the projector during class. Both run on Universal-3.5 Pro.

What is a lecture capture system?

A lecture capture system records classroom audio and video, then makes those recordings available for students to review later. That's the whole job in one sentence. The interesting part is what you do with the recording once you have it.

Speaker diarization is what turns a recording into something usable. It answers "who spoke when," which means a transcript stops being a wall of text and becomes navigable. A student who missed Thursday can jump to the three minutes where the professor explained the thing they don't understand, instead of scrubbing through a 75-minute file.

Four things get better once speaker labels exist:

• Search that works. Students can search within a speaker, not just within a transcript.
• Captions that name the speaker. Required for accessibility, useful for everyone.
• Skippable Q&A. Lecture content and student questions become separable.
• Study notes that read like a conversation. Because they were one.

Why speaker diarization matters for lecture capture

A transcript without speaker labels is a search index. A transcript with them is a study tool.

The difference shows up most for the students who need it most. International learners can anchor on who is speaking while they work through an unfamiliar accent. Screen readers can announce speaker changes. And any downstream tool you build — a summarizer, a study-guide generator, a question extractor — gets dramatically better input when it knows the professor's words from a student's.

A note on privacy before you write any code. Diarization returns anonymous labels: Speaker A, Speaker B. It does not know who anyone is, and in a classroom that's a feature. Several privacy frameworks — FERPA in the US chief among them — counsel real care around creating and storing identifiable records of individual student voices. The approach this tutorial takes: generic labels for students, role labels at most, informed consent before recording, and access controls on the transcript. There's an implementation section on this below.

Lecture capture system architecture options

You can buy a hardware lecture capture appliance or you can build a software system on a computer with a decent microphone. The choice mostly determines how good your speaker labels get, because diarization quality is a model problem, not a microphone problem.

Software-based systems for speaker diarization

Software wins on diarization for a boring reason: you can swap the model. A hardware appliance ships with whatever speaker separation its vendor built, and you get the next version when you buy the next box. A software system calls an API, and the model improves underneath you.

For reference on the cost side: AssemblyAI's pre-recorded transcription runs $0.21/hr on Universal-3.5 Pro with speaker diarization at +$0.02/hr, and the free tier covers 185 hours of pre-recorded audio. A 15-week course with three 75-minute lectures a week is about 56 hours of audio.

How to build a lecture capture system with speaker diarization

Four pieces: capture the audio, transcribe it with speaker labels, attribute the speakers, and write out captions. We'll build each as its own module.

Set up audio capture for speaker diarization

Diarization works by building a voice embedding for each speaker and clustering on it. Everything that muddies those embeddings — noise, cross-talk, wildly uneven volume — costs you accuracy. So the audio setup matters more here than it does for plain transcription.

Record at 16 kHz minimum, 16-bit, mono. Higher sample rates don't hurt but they don't help diarization much either; clean separation between voices helps far more than extra bandwidth. The docs' guidance is worth internalizing: each speaker should get at least 30 seconds of speech before you should expect a stable label for them. A student who says "yeah" once is not going to get their own cluster, and that's correct behavior.

Test your room before you trust it:

# mic_test.py
import wave

import sounddevice as sd

SAMPLE_RATE = 16000  # the minimum for reliable diarization


def test_microphone_quality(duration=10):
    """Record a short two-voice sample so you can listen for problems."""
    print(f"Recording {duration} seconds...")
    print("Speak normally, then have someone else speak from across the room.")

    recording = sd.rec(
        int(duration * SAMPLE_RATE),
        samplerate=SAMPLE_RATE,
        channels=1,
        dtype="int16",
    )
    sd.wait()

    with wave.open("mic_test.wav", "wb") as wf:
        wf.setnchannels(1)
        wf.setsampwidth(2)
        wf.setframerate(SAMPLE_RATE)
        wf.writeframes(recording.tobytes())

    print("Saved mic_test.wav — listen before you go further.")


if __name__ == "__main__":
    test_microphone_quality()

Listen for four things: are both voices clearly distinct, is the HVAC audible, does the volume collapse when someone speaks from the back, and is there slap echo off a hard wall. Fix those in the room. No model fixes them for you — see the hard cases in speaker diarization for what happens when you don't.

Configure your Python environment

pip install assemblyai pyaudio sounddevice python-dotenv

assemblyai needs to be 1.0 or newer for the assemblyai.prerecorded.v2 namespace used below. One gotcha: pyaudio builds against PortAudio, so on macOS you'll want brew install portaudio first, and on Debian or Ubuntu sudo apt install portaudio19-dev.

Lay the project out like this:

lecture-capture/
├── .env                     # ASSEMBLYAI_API_KEY=...
├── config.py                # audio + key configuration
├── recorder.py              # capture the room
├── processor.py             # speaker diarization
├── captions.py              # WebVTT, SRT, searchable text
├── live_captions.py         # streaming diarization
├── main.py                  # the async pipeline, end to end
└── recordings/              # audio, transcripts, captions

Configuration first. Read the key from the environment; never put it in source:

# config.py
import os

from dotenv import load_dotenv

load_dotenv()


class AudioConfig:
    # Recording quality for speaker diarization
    SAMPLE_RATE = 16000
    CHANNELS = 1
    CHUNK_SIZE = 1024

    # API key for speech processing
    ASSEMBLYAI_API_KEY = os.environ["ASSEMBLYAI_API_KEY"]

    # File organization
    RECORDINGS_DIR = "recordings"

    # Default lecture length, in seconds
    DEFAULT_DURATION = 3600

Record the lecture

Standard PyAudio callback recorder, with a timer so it stops on its own if everyone forgets:

Transcribe with speaker diarization

Here's the part that does the actual work. Set speaker_labels=True and the transcript comes back as an array of utterances — one uninterrupted stretch of speech from one speaker — instead of a single block of text.

Two things worth knowing before you read the code.

First, the speaker count parameters are hard boundaries, not hints. max_speakers_expected is a strict cap: if eleven people speak and you set it to four, speakers five through eleven get merged into the four labels you allowed. And min_speakers_expected is a strict floor, which will happily split one person into three if you set it too high. The defaults are duration-based and sensible for lectures — no cap under 2 minutes, 10 speakers for 2 to 10 minutes, and 30 speakers for anything over 10 minutes. A full lecture lands in that last bucket, so the honest advice is to leave the range unset unless you know your room.

Second, pin your model. Passing speech_models=["universal-3-5-pro", "universal-2"] gets you Universal-3.5 Pro — 18 languages natively, with the most accurate diarization we've shipped — and automatic fallback to Universal-2 for anything outside that set, which covers 99 languages in total. That fallback matters more in a university than almost anywhere else. If you'd rather see how multilingual rooms behave in practice, there's a separate walkthrough on detecting, diarizing, and transcribing across languages.

# recorder.py
import os
import threading
import wave
from datetime import datetime

import pyaudio

from config import AudioConfig


class LectureRecorder:
    def __init__(self):
        self.audio = pyaudio.PyAudio()
        self.stream = None
        self.frames = []
        self.recording = False

        os.makedirs(AudioConfig.RECORDINGS_DIR, exist_ok=True)

    def start_recording(self, duration=None):
        """Begin recording classroom audio."""
        self.recording = True
        self.frames = []

        self.stream = self.audio.open(
            format=pyaudio.paInt16,
            channels=AudioConfig.CHANNELS,
            rate=AudioConfig.SAMPLE_RATE,
            input=True,
            frames_per_buffer=AudioConfig.CHUNK_SIZE,
            stream_callback=self._audio_callback,
        )

        print("Recording started...")
        self.stream.start_stream()

        if duration:
            threading.Timer(duration, self.stop_recording).start()
            print(f"Will stop automatically after {duration / 60:.1f} minutes")

    def _audio_callback(self, in_data, frame_count, time_info, status):
        """Handle incoming audio data."""
        if self.recording:
            self.frames.append(in_data)
        return (in_data, pyaudio.paContinue)

    def stop_recording(self):
        """Stop recording and save the audio file."""
        if not self.recording:
            return None

        self.recording = False
        self.stream.stop_stream()
        self.stream.close()

        timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
        filename = f"{AudioConfig.RECORDINGS_DIR}/lecture_{timestamp}.wav"

        with wave.open(filename, "wb") as wf:
            wf.setnchannels(AudioConfig.CHANNELS)
            wf.setsampwidth(self.audio.get_sample_size(pyaudio.paInt16))
            wf.setframerate(AudioConfig.SAMPLE_RATE)
            wf.writeframes(b"".join(self.frames))

        recorded_seconds = (
            len(self.frames) * AudioConfig.CHUNK_SIZE / AudioConfig.SAMPLE_RATE
        )
        print(f"Recording saved: {filename}")
        print(f"Duration: {recorded_seconds:.1f} seconds")

        return filename

    def __del__(self):
        if hasattr(self, "audio"):
            self.audio.terminate()

Note the speech_model_used field on the response. It tells you which model actually ran, which is how you find out that last Tuesday's guest lecture in Portuguese quietly routed to Universal-2.

Put roles on the speakers, carefully

"Speaker A" is safe but not especially useful. The natural next step is naming, and this is exactly where a lecture capture system should slow down.

Speaker Identification maps the generic labels to real names or roles. You give it the labels you expect and it returns a mapping — {"A": "Instructor", "B": "Student"} — and rewrites utterances[].speaker for you. It costs $0.02/hr.

Use speaker_type: "role", not "name". A role label tells a student everything they need ("the professor said this, a classmate said that") and creates no record tying a specific person's voice to their words. That distinction is the whole ballgame under FERPA. If you're fuzzy on the terminology here, diarization vs. recognition vs. identification is worth ten minutes.

# Roles, not names. "Instructor" is useful to a student; a student's name is a
# record you probably shouldn't be creating.
results = processor.identify_speakers(
    audio_file,
    speaker_roles=["Instructor", "Student"],
)

Note the speech_model_used field on the response. It tells you which model actually ran, which is how you find out that last Tuesday's guest lecture in Portuguese quietly routed to Universal-2.

Put roles on the speakers, carefully

"Speaker A" is safe but not especially useful. The natural next step is naming, and this is exactly where a lecture capture system should slow down.

Speaker Identification maps the generic labels to real names or roles. You give it the labels you expect and it returns a mapping — {"A": "Instructor", "B": "Student"} — and rewrites utterances[].speaker for you. It costs $0.02/hr.

Use speaker_type: "role", not "name". A role label tells a student everything they need ("the professor said this, a classmate said that") and creates no record tying a specific person's voice to their words. That distinction is the whole ballgame under FERPA. If you're fuzzy on the terminology here, diarization vs. recognition vs. identification is worth ten minutes.

Live captions with streaming diarization

Everything so far assumes the lecture is over. For captions on the projector while class is happening, you want the streaming API instead — and streaming diarization is supported on all three streaming models, so enabling it is one parameter — see the Streaming Diarization documentation for the full message shapes.

It's a genuinely harder problem than async diarization, and it's worth being clear about why. Async diarization sees the whole recording before it decides anything. Streaming diarization has to label a turn the moment it ends, with only the audio it has heard so far. Three consequences you have to design around:

• Turns under about a second come back as "PENDING". There isn't enough audio to build a reliable embedding, and guessing would split one speaker across several labels. Show the text, hold the label.
• The first one or two turns of a session may be wrong. The model hasn't built speaker profiles yet. It self-corrects quickly.
• Overlapping speech gets assigned to one speaker. The model can't split simultaneous voices. Frequent cross-talk degrades everything.

The fix for the first two is SpeakerRevision. When the session ends, the server does a final refinement pass with the whole conversation in view and sends back corrections for any turn whose label changed — one message, about 400 ms, arriving before termination. So you get fast labels during class and correct labels for the file you save. There's more on the design of that in One stream, two jobs.

Here's a complete live-captioning script that handles all of it:

# live_captions.py
import pyaudio
from assemblyai.streaming.v3 import (
    BeginEvent,
    SpeakerRevisionEvent,
    StreamingClient,
    StreamingClientOptions,
    StreamingError,
    StreamingEvents,
    StreamingParameters,
    TerminationEvent,
    TurnEvent,
)

from config import AudioConfig

# Turns are stored by turn_order so end-of-session revisions can correct them.
turns_by_order = {}


def microphone_frames(chunk_size=800):
    """Yield raw 16-bit PCM frames from the default input device."""
    audio = pyaudio.PyAudio()
    stream = audio.open(
        format=pyaudio.paInt16,
        channels=1,
        rate=AudioConfig.SAMPLE_RATE,
        input=True,
        frames_per_buffer=chunk_size,
    )
    try:
        while True:
            yield stream.read(chunk_size, exception_on_overflow=False)
    finally:
        stream.stop_stream()
        stream.close()
        audio.terminate()


def on_begin(client, event: BeginEvent):
    print(f"Session {event.id} open. Model: {event.configuration.speech_model}")
    print("Live captions below. Press Ctrl+C to end the session.\n")


def on_turn(client, event: TurnEvent):
    if not event.transcript:
        return

    if not event.end_of_turn:
        # Partial. Overwrite the same line so captions read as one live caption.
        print(f"\r  ... {event.transcript[-90:]}", end="", flush=True)
        return

    turns_by_order[event.turn_order] = event

    # Turns shorter than ~1 second come back as PENDING: the model does not yet
    # have enough audio for a reliable speaker embedding. Show the text anyway.
    label = event.speaker_label
    speaker = "Speaker ?" if label in (None, "PENDING") else f"Speaker {label}"

    print(f"\r{' ' * 100}\r[{speaker}] {event.transcript}")


def on_speaker_revision(client, event: SpeakerRevisionEvent):
    """Apply the end-of-session refinement pass before saving the transcript."""
    print(f"\nRevising speaker labels on {len(event.revisions)} turn(s).")
    for revision in event.revisions:
        turn = turns_by_order.get(revision.turn_order)
        if turn is None:
            continue
        turn.speaker_label = revision.speaker_label
        for word, revised in zip(turn.words, revision.words):
            word.speaker = revised.speaker


def on_terminated(client, event: TerminationEvent):
    print(f"\nSession ended. {event.audio_duration_seconds}s of audio processed.")
    print("\nFinal transcript with revised speaker labels:\n")
    for turn_order in sorted(turns_by_order):
        turn = turns_by_order[turn_order]
        label = turn.speaker_label
        speaker = "Speaker ?" if label in (None, "PENDING") else f"Speaker {label}"
        print(f"[{speaker}] {turn.transcript}")


def on_error(client, error: StreamingError):
    print(f"\nStreaming error: {error}")


def main():
    client = StreamingClient(
        StreamingClientOptions(api_key=AudioConfig.ASSEMBLYAI_API_KEY)
    )

    client.on(StreamingEvents.Begin, on_begin)
    client.on(StreamingEvents.Turn, on_turn)
    client.on(StreamingEvents.SpeakerRevision, on_speaker_revision)
    client.on(StreamingEvents.Termination, on_terminated)
    client.on(StreamingEvents.Error, on_error)

    client.connect(
        StreamingParameters(
            sample_rate=AudioConfig.SAMPLE_RATE,
            speech_model="universal-3-5-pro",
            speaker_labels=True,
            # Hard cap, not a hint. Leave headroom over the voices you expect;
            # too high and the model over-splits one speaker into several.
            max_speakers=4,
            # Lecture pauses are longer than a phone call's. Give a speaker
            # room to breathe before the turn is closed.
            min_turn_silence=560,
            max_turn_silence=2000,
        )
    )

    try:
        client.stream(microphone_frames())
    except KeyboardInterrupt:
        print("\nStopping...")
    finally:
        client.disconnect(terminate=True)


if __name__ == "__main__":
    main()

Two parameters there deserve a note. max_speakers accepts 1 through 10 and behaves like its async cousin — a hard cap, so leave headroom. And min_turn_silence / max_turn_silence control turn detection, which on Universal-3.5 Pro is punctuation-based rather than confidence-based. The defaults are tuned for voice agents, where people interrupt; a lecture hall wants longer, so 560 ms and 2000 ms are better starting points than the defaults. If you're migrating from an older streaming model, note that format_turns and end_of_turn_confidence_threshold don't apply here — formatting is always on, and turn detection changed.

Streaming diarization bills at +$0.12/hr on top of the $0.45/hr streaming rate — the full breakdown is on the pricing page.

Generate speaker-attributed captions

Now turn that JSON into files a video player and a learning management system can actually use.

WebVTT has a feature most tutorials skip: the voice span, <v Speaker A>. Players that understand it style the speaker separately from the caption text; players that don't just render the name. Either way the speaker survives into the caption, which is the entire point and something a plain text-only caption loses.

# captions.py
import json


def _split_timestamp(milliseconds):
    """Break a millisecond offset into (hours, minutes, seconds, milliseconds)."""
    total_ms = int(round(milliseconds))
    hours, remainder = divmod(total_ms, 3_600_000)
    minutes, remainder = divmod(remainder, 60_000)
    seconds, ms = divmod(remainder, 1000)
    return hours, minutes, seconds, ms


def to_webvtt_timestamp(milliseconds):
    """HH:MM:SS.mmm"""
    h, m, s, ms = _split_timestamp(milliseconds)
    return f"{h:02d}:{m:02d}:{s:02d}.{ms:03d}"


def to_srt_timestamp(milliseconds):
    """HH:MM:SS,mmm"""
    h, m, s, ms = _split_timestamp(milliseconds)
    return f"{h:02d}:{m:02d}:{s:02d},{ms:03d}"


def to_readable_timestamp(milliseconds):
    """H:MM:SS, for a study-notes transcript."""
    h, m, s, _ = _split_timestamp(milliseconds)
    return f"{h}:{m:02d}:{s:02d}"


class CaptionGenerator:
    def _load(self, transcript_file):
        with open(transcript_file, "r", encoding="utf-8") as f:
            return json.load(f)

    def create_webvtt_captions(self, transcript_file, output_file):
        """Generate WebVTT captions with speaker labels as voice spans."""
        transcript = self._load(transcript_file)

        lines = ["WEBVTT", ""]

        for i, segment in enumerate(transcript["speaker_segments"], 1):
            start = to_webvtt_timestamp(segment["start_time_ms"])
            end = to_webvtt_timestamp(segment["end_time_ms"])

            lines.append(str(i))
            lines.append(f"{start} --> {end}")
            # <v Speaker A> is the WebVTT voice span. Players that understand it
            # style the speaker; players that don't fall back to showing the name.
            lines.append(f"<v {segment['speaker']}>{segment['text']}")
            lines.append("")

        with open(output_file, "w", encoding="utf-8") as f:
            f.write("\n".join(lines))

        print(f"WebVTT captions saved: {output_file}")
        return output_file

    def create_srt_captions(self, transcript_file, output_file):
        """Generate SRT captions as an alternative format."""
        transcript = self._load(transcript_file)

        lines = []

        for i, segment in enumerate(transcript["speaker_segments"], 1):
            start = to_srt_timestamp(segment["start_time_ms"])
            end = to_srt_timestamp(segment["end_time_ms"])

            lines.append(str(i))
            lines.append(f"{start} --> {end}")
            lines.append(f"[{segment['speaker']}] {segment['text']}")
            lines.append("")

        with open(output_file, "w", encoding="utf-8") as f:
            f.write("\n".join(lines))

        print(f"SRT captions saved: {output_file}")
        return output_file

    def create_searchable_transcript(self, transcript_file, output_file):
        """Generate a text file students can search, organized two ways."""
        transcript = self._load(transcript_file)

        speaker_content = {}
        for segment in transcript["speaker_segments"]:
            speaker_content.setdefault(segment["speaker"], []).append(segment["text"])

        with open(output_file, "w", encoding="utf-8") as f:
            f.write("LECTURE TRANSCRIPT WITH SPEAKER DIARIZATION\n")
            f.write("=" * 50 + "\n\n")

            f.write("TIMELINE VIEW:\n")
            f.write("-" * 20 + "\n")
            for segment in transcript["speaker_segments"]:
                timestamp = to_readable_timestamp(segment["start_time_ms"])
                f.write(f"[{timestamp}] {segment['speaker']}: {segment['text']}\n")

            f.write("\n\nSPEAKER SUMMARY:\n")
            f.write("-" * 20 + "\n")
            for speaker, content in speaker_content.items():
                f.write(f"\n{speaker}:\n")
                f.write("\n".join(content))
                f.write("\n")

        print(f"Searchable transcript saved: {output_file}")
        return output_file

One detail that bites people: those timestamp helpers use integer divmod on the raw millisecond value rather than datetime.timedelta. A 75-minute lecture crosses the one-hour mark, and a naive implementation that reads timedelta.seconds silently drops the hour. Run a caption from the back half of a long file and check that it says 01: and not 00:.

Wire it together

# main.py
from captions import CaptionGenerator
from processor import SpeakerProcessor
from recorder import LectureRecorder


def main():
    print("Lecture capture system with speaker diarization")
    print("-" * 50)

    recorder = LectureRecorder()
    processor = SpeakerProcessor()
    caption_gen = CaptionGenerator()

    try:
        duration_input = input("Recording duration in minutes (Enter for 60): ")
        duration_minutes = int(duration_input) if duration_input.strip() else 60
    except ValueError:
        duration_minutes = 60
    duration_seconds = duration_minutes * 60

    audio_file = None
    try:
        recorder.start_recording(duration=duration_seconds)
        input("Press Enter to stop recording early...\n")
        audio_file = recorder.stop_recording()
    except KeyboardInterrupt:
        print("\nRecording stopped by user")
        if recorder.recording:
            audio_file = recorder.stop_recording()

    if not audio_file:
        print("No recording was saved.")
        return

    # Leave expected_speakers unset to use the duration-based defaults, or pass
    # a range like (2, 8) for a seminar where you know roughly who talks.
    results = processor.identify_speakers(audio_file)

    transcript_file = audio_file.replace(".wav", "_transcript.json")
    processor.save_transcript(results, transcript_file)

    base_filename = audio_file.replace(".wav", "")
    caption_gen.create_webvtt_captions(transcript_file, f"{base_filename}.vtt")
    caption_gen.create_srt_captions(transcript_file, f"{base_filename}.srt")
    caption_gen.create_searchable_transcript(
        transcript_file, f"{base_filename}_searchable.txt"
    )

    print("\nProcessing complete.")
    print(f"Model used: {results['model_used']}")
    print(f"Speakers identified: {len(results['speakers_detected'])}")
    print(f"Total duration: {results['audio_duration_seconds'] / 60:.1f} minutes")


if __name__ == "__main__":
    main()

Validate before you trust it

Diarization either works on your rooms or it doesn't, and you find out empirically. Three checks worth running on a real lecture, not a test clip:

1. Count the labels. More speakers than people in the room means over-splitting — usually noise or a too-high min_speakers_expected. Fewer means merging, usually short contributions or a too-low cap.
2. Spot-check the Q&A. It's the hardest part of any lecture recording and the part students use most.
3. Measure it, if you're deploying at scale. There's a real metric for this, and how to measure speaker diarization accuracy with cpWER in Python walks through it. Guessing from vibes across 40 rooms doesn't scale.

Technical requirements and compliance considerations

Audio quality targets, restated as a checklist: 16 kHz sample rate minimum, 16-bit depth or better, background noise low enough that voices stay distinct, consistent levels from every seat you expect speech from, and at least 30 seconds of speech per person you want labeled.

Privacy is the part that gets systems shut down, so treat it as a design constraint rather than a disclaimer. Three frameworks usually apply. FERPA (US) protects students' educational records and restricts unauthorized disclosure, which is the direct argument against building a persistent voice-to-name record of individual students. ADA and Section 508 (for federal institutions) require accessible content — speaker-labeled captions help satisfy this. GDPR (EU) requires a lawful basis and data minimization for processing identifiable personal data, and voice recordings qualify.

That last row is why processor.py prints the transcript ID on every run. You need it to delete a transcript later, and "we can't find it" is not an answer to a deletion request.

On institutional agreements: AssemblyAI enables covered entities and their business associates subject to HIPAA to use the AssemblyAI services to process protected health information (PHI). AssemblyAI is considered a business associate under HIPAA, and offers a standard Business Associate Addendum (BAA). That's relevant if your recordings touch a teaching hospital or a clinical program; for ordinary coursework it isn't.

For accessibility, the speaker-labeled captions this pipeline produces support Section 508 and ADA captioning requirements for recorded content. Live captioning during class is what live_captions.py is for.

Final words

The system in this post is four modules and about 400 lines, and the interesting design decision isn't in any of them. It's the choice to stop at roles.

Everything technical here pushes toward more identification: diarization separates the voices, Speaker Identification can name them, and the API will happily accept a roster. But a lecture capture system that knows which student asked which question has created a record that a system labeling them "Student" has not — and the second system is just as useful to the person the tool is actually for. The constraint makes the product better. That's rarer than it sounds.

If you're extending this: the searchable transcript is the obvious hook for a summarizer, and role labels make a much better prompt than Speaker A ever did.

Frequently asked questions

Which Python packages do I need for lecture capture with speaker diarization?

Four: assemblyai (1.0 or newer, for the assemblyai.prerecorded.v2 namespace), pyaudio for recording, sounddevice for the microphone test, and python-dotenv to load your API key. pyaudio compiles against PortAudio, so install that first — brew install portaudio on macOS, sudo apt install portaudio19-dev on Debian or Ubuntu.

What's the right max_speakers_expected value for a lecture?

Usually none. max_speakers_expected is a hard cap, not a hint, and the duration-based defaults already suit lectures: 30 speakers for any file over 10 minutes. Set a range only if you know your room — and when you do, set max_speakers_expected a couple above the count you expect, because setting it too high causes over-splitting and setting it too low merges people together. Use speakers_expected only when you're certain of the exact number; a wrong exact count hurts accuracy more than no constraint at all.

How accurate is speaker diarization in a real classroom?

It depends on the audio, and the biggest lever is how much each person speaks. The model builds a voice embedding per speaker and clusters on it, so accuracy improves the longer someone talks — the docs suggest at least 30 seconds of speech per speaker. Cross-talk, background noise, and one-word contributions are what degrade it. A professor and a handful of question-askers is a well-behaved case; a seminar where twelve people interrupt each other is not.

Can I identify speakers by name instead of "Speaker A"?

Yes, with Speaker Identification (+$0.02/hr), which maps the generic labels to names or roles. In an educational setting, use speaker_type: "role" rather than "name". Role labels like "Instructor" and "Student" give students everything they need without creating a record that links an individual voice to an identity — which is the specific thing FERPA counsels care around. Check with your institution's privacy office before naming anyone.

Is real-time speaker diarization available for live lectures?

Yes. Add speaker_labels: true to your streaming connection parameters; it's supported on all three streaming models, including universal-3-5-pro. Three limitations to design around: turns under about a second are labeled "PENDING" rather than guessed, the first one or two turns of a session may be misassigned while the model builds speaker profiles, and overlapping speech gets attributed to a single speaker. The end-of-session SpeakerRevision message fixes the first two in the saved transcript.

Can I process lecture recordings I already have?

Yes. Any audio or video file at 16 kHz or above works — pass a local path or a URL to transcriber.transcribe() and the SDK handles the upload. Nothing about the async path depends on when the recording was made, which makes back-cataloging an existing archive the cheapest win available to most departments.

What does this cost to run for a semester?

Pre-recorded transcription on Universal-3.5 Pro is $0.21/hr with diarization at +$0.02/hr, so a 15-week course with three 75-minute lectures a week — roughly 56 hours — runs about $13. Role-based Speaker Identification adds $0.02/hr. Live captioning is priced differently: $0.45/hr for streaming plus $0.12/hr for streaming diarization. The free tier covers 185 hours of pre-recorded audio and 333 hours of streaming, which is enough to pilot a couple of courses before you pay anything.

How should I handle student privacy in a diarized lecture recording?

Five things, in order of how often they get skipped: use generic or role labels rather than names, get written consent before recording and put a notice in the syllabus, keep the API key in an environment variable rather than source control, set an automatic deletion policy tied to the end of term, and log every transcript ID so you can honor a deletion request. FERPA restricts unauthorized disclosure of student educational records — talk to your institution's privacy office before deploying anything that creates a persistent, identifiable record of student voices.

‍