AI GitHub
WhisperX for word-aligned transcripts
Add word timing and optional speaker diarization to a transcription workflow.
Key Takeaways
- WhisperX combines speech recognition with a separate alignment stage.
- Prepare the recognition, alignment and optional diarization models as separate dependencies.
- Use a two-speaker clip with several interruptions.
- Alignment needs a model compatible with the spoken language.
How it works
WhisperX combines speech recognition with a separate alignment stage. Diarization can assign speaker labels when the configured models and audio support it.
What you need
Prepare the recognition, alignment and optional diarization models as separate dependencies. Use a language with documented alignment support for your first test.
What to try first
Use a two-speaker clip with several interruptions. Compare aligned words with a waveform and check how labels behave when speakers overlap.
Save recognition and alignment outputs separately. If the text is right but a word timestamp is wrong, the distinction tells you which stage to investigate.
Where to look for failures
Alignment needs a model compatible with the spoken language. Speaker labels identify clusters rather than verified identities, and overlapping speech can still confuse the pipeline.
Related Franklin pages
Check whether the task needs speech detection, precise word timing or speaker segments. Save the output of each stage so a timing error can be traced to its origin.
- Run speaker diarization pipelines to identify who spoke when in a recording. pyannote.audio for separating speakers by time
- Detect intervals likely to contain speech in local audio processing. Silero VAD for locating speech segments
For an application where the quality of recording and handling audio matters, see Franklin’s Open Mic AI profile. Open Mic AI
Project documentation
Try it for your workflow
Explore the project’s documentation for setup and usage, or save this page to a collection for later.
Read the project documentation ↗