AI GitHub
MLX LM for language models on Macs
Generate with, quantize and adapt supported language models using MLX.
Key Takeaways
- MLX LM provides language-model utilities on top of MLX, including checkpoint conversion, generation and fine-tuning workflows.
- Choose either a preconverted model or a checkpoint the conversion tools support.
- Generate from an existing MLX-format checkpoint before converting one yourself.
- Memory is shared across applications, and the prompt cache grows with context.
How it works
MLX LM provides language-model utilities on top of MLX, including checkpoint conversion, generation and fine-tuning workflows. It lets a developer work with model files directly on an Apple silicon machine.
What you need
Choose either a preconverted model or a checkpoint the conversion tools support. Record tokenizer and chat template settings alongside weights when comparing published variants.
What to try first
Generate from an existing MLX-format checkpoint before converting one yourself. Compare the original and quantized versions on several prompts with the same chat template.
Compare generation with fresh context and cached context. An app that keeps the prompt cache may behave differently from independent command-line invocations.
Where to look for failures
Memory is shared across applications, and the prompt cache grows with context. Test on the actual target Mac and check support for the checkpoint architecture before downloading a large model.
Related Franklin pages
The checkpoint and its runtime are separate choices. Select a model small enough for the device, then find a compatible format and test both the response and memory headroom.
- Run quantized language models with a native CLI or HTTP server. llama.cpp for GGUF inference
- Work with array operations, automatic differentiation and model execution on Apple silicon. MLX for machine learning on Apple silicon
For the constraints that shape mobile and embedded model deployment, see Franklin’s guide. On-device AI
Project documentation
Try it for your workflow
Explore the project’s documentation for setup and usage, or save this page to a collection for later.
Read the project documentation ↗