AI GitHub
whisper.cpp for local Whisper transcription
Run compatible Whisper models through a native inference engine.
Key Takeaways
- whisper.cpp provides a portable implementation that can use converted model files and supported hardware backends.
- Start with a converted model and a build for the device or accelerator.
- Transcribe the same clip on CPU and an available accelerator.
- Build flags and model format affect what the runtime can use.
How it works
whisper.cpp provides a portable implementation that can use converted model files and supported hardware backends. Its examples show how to integrate local recognition into different environments.
What you need
Start with a converted model and a build for the device or accelerator. Confirm the audio format expected by the example or library binding you plan to use.
What to try first
Transcribe the same clip on CPU and an available accelerator. Compare speed and timestamps with the reference model before changing quantization.
Measure performance after loading the model. A mobile or embedded process may have a larger startup penalty than the transcription time for a short clip.
Where to look for failures
Build flags and model format affect what the runtime can use. Test on the target machine, especially for live audio where sustained speed matters.
Related Franklin pages
Use the same source audio and decoding assumptions for transcription tests. Measure performance and listen to the mistakes, especially around silence, names and language changes.
- Run Whisper recognition through CTranslate2 with configurable compute precision. faster-whisper for CTranslate2 transcription
- Embed supported speech models in applications through ONNX-based runtimes. sherpa-onnx for local speech applications
For an application where the quality of recording and handling audio matters, see Franklin’s Open Mic AI profile. Open Mic AI
Project documentation
Try it for your workflow
Explore the project’s documentation for setup and usage, or save this page to a collection for later.
Read the project documentation ↗