Small language model
A compact language model that runs on a laptop or phone, trading some breadth for speed, cost and privacy.
What it is
Small language models (SLMs) work like large ones but with far fewer parameters, typically a few billion or less. They are good enough for focused jobs such as classification, extraction, autocomplete or on-device assistants, and they can run without sending data to the cloud.
How it works
They use the same transformer design as larger models. Much of their quality comes from careful data choices and from knowledge distillation, where a large model teaches the small one. Quantization then shrinks the weights further so they fit in a phone or laptop memory.
What it is good for
- On-device and offline assistants
- Private processing of sensitive text
- Cheap, fast classification and extraction at scale
- Fine-tuning for one narrow task
Limits to know
- Less world knowledge and weaker multi-step reasoning
- More sensitive to prompt wording
- Often needs fine-tuning to match a large model on a task
