SmolLM for exploring compact language models

How it works

The repository organizes artifacts and references for the SmolLM family. Each release links checkpoints to documentation about training, supported use and evaluation.

What you need

The repository includes artifacts for different capabilities and sizes. Choose a text or multimodal checkpoint according to your task, then follow its model card for preprocessing and runtime support.

What to try first

Compare two published sizes on the same short tasks. Test both task accuracy and the memory needed to run each model in a compatible inference environment.

Include one classification task with deliberately similar labels and one short summarization task. A small model can fit comfortably on the machine while failing the task you care about.

Where to look for failures

A compact checkpoint can still struggle with long prompts or unfamiliar domains. Check the model card for the variant and use a fixed evaluation set before choosing a size.

Related Franklin pages

The checkpoint and its runtime are separate choices. Select a model small enough for the device, then find a compatible format and test both the response and memory headroom.

• Generate with, quantize and adapt supported language models using MLX. MLX LM for language models on Macs
• Run quantized language models with a native CLI or HTTP server. llama.cpp for GGUF inference

For more on how model size affects deployment decisions, read Franklin’s guide. Small language models

Project documentation

• Current project README
• README at the reviewed revision