AI GitHub
MLX for machine learning on Apple silicon
Work with array operations, automatic differentiation and model execution on Apple silicon.
Key Takeaways
- MLX uses lazy evaluation and a unified memory model.
- Begin with a small operation in the target language bindings and confirm device placement.
- Implement a small matrix operation and a training step, then force evaluation before timing them.
- Deferred execution can make a benchmark measure graph construction instead of completed work.
How it works
MLX uses lazy evaluation and a unified memory model. Arrays can be used by different supported devices without the same explicit transfer model common in discrete GPU stacks.
What you need
Begin with a small operation in the target language bindings and confirm device placement. Workloads that repeatedly cross library boundaries should include those transfers in timing.
What to try first
Implement a small matrix operation and a training step, then force evaluation before timing them. Compare runs with identical tensor shapes and data types.
Try a long-lived process that repeatedly allocates and releases large arrays. A brief timing result may miss memory pressure that appears in an interactive workload.
Where to look for failures
Deferred execution can make a benchmark measure graph construction instead of completed work. Unified memory is shared by the rest of the system, so monitor memory pressure while loading models.
Related Franklin pages
Decide whether you need array programming, a language-model workflow or an embeddable library. That choice determines how much model and input handling your application must implement.
- Generate with, quantize and adapt supported language models using MLX. MLX LM for language models on Macs
- Build native Rust programs around tensors and model implementations. Candle for embedding inference in Rust
For the constraints that shape mobile and embedded model deployment, see Franklin’s guide. On-device AI
Project documentation
Try it for your workflow
Explore the project’s documentation for setup and usage, or save this page to a collection for later.
Read the project documentation ↗