Speech-to-text model
Turns spoken audio into written text, often in many languages and with timestamps.
What it is
Speech-to-text (automatic speech recognition) models transcribe recordings and live audio. Modern ones handle accents, background noise and dozens of languages, and many can also translate speech or mark who is speaking.
How it works
Audio is converted into a spectrogram, a picture of sound frequencies over time. An encoder turns it into features and a decoder writes the matching text token by token. Training on large amounts of audio paired with transcripts teaches robustness to noise and accents.
What it is good for
- Meeting notes and subtitles
- Voice input for apps and assistants
- Searching podcasts and calls
- Accessibility captions
Limits to know
- Struggles with rare names, jargon and overlapping speakers
- Can invent words in long silences
- Real-time use needs a fast model or GPU
