AI GitHub
pyannote.audio for separating speakers by time
Run speaker diarization pipelines to identify who spoke when in a recording.
Key Takeaways
- Diarization estimates speaker activity and associates spans with speaker clusters.
- Choose a documented diarization pipeline and check the model's access requirements.
- Test a clip with a known number of speakers and another with overlaps.
- Some models require accepting access conditions.
How it works
Diarization estimates speaker activity and associates spans with speaker clusters. The result can be combined with a transcript from a separate recognizer.
What you need
Choose a documented diarization pipeline and check the model's access requirements. Keep time units and segment boundaries consistent when joining speaker information with a transcript.
What to try first
Test a clip with a known number of speakers and another with overlaps. Compare speaker changes with a hand-marked reference before merging labels into text.
Assign a visible unknown or overlapping label in your fixture. A forced single-speaker assignment can hide uncertainty that matters when editing or reviewing a recording.
Where to look for failures
Some models require accepting access conditions. Microphone quality, short utterances and overlapping speech affect speaker assignments.
Related Franklin pages
Check whether the task needs speech detection, precise word timing or speaker segments. Save the output of each stage so a timing error can be traced to its origin.
- Detect intervals likely to contain speech in local audio processing. Silero VAD for locating speech segments
- Add word timing and optional speaker diarization to a transcription workflow. WhisperX for word-aligned transcripts
For an application where the quality of recording and handling audio matters, see Franklin’s Open Mic AI profile. Open Mic AI
Project documentation
Try it for your workflow
Explore the project’s documentation for setup and usage, or save this page to a collection for later.
Read the project documentation ↗