AI GitHub
Unsloth for memory-conscious model tuning
Fine-tune supported model architectures using optimized training paths and documented recipes.
Key Takeaways
- Unsloth provides model loading and fine-tuning workflows with optimizations for supported configurations.
- Follow the installation instructions for your accelerator and chosen model.
- Start with a documented small model and record peak memory and examples per second.
- Performance claims depend on hardware, sequence length and training setup.
How it works
Unsloth provides model loading and fine-tuning workflows with optimizations for supported configurations. Checkpoint choice and precision determine which path is available.
What you need
Follow the installation instructions for your accelerator and chosen model. Select the training precision and output format with the eventual inference runtime in mind.
What to try first
Start with a documented small model and record peak memory and examples per second. Compare resulting outputs on a fixed holdout before exporting the checkpoint.
Record throughput at the sequence lengths present in your data. A short sample can underestimate memory use and oversell the settings that fit.
Where to look for failures
Performance claims depend on hardware, sequence length and training setup. Confirm the export format works in the runtime that will serve the model.
Related Franklin pages
Run a tiny supervised job and check the prepared examples before investing in a large run. Save the base checkpoint, data revision and validation outputs with the recipe.
- Adapt language-model tuning recipes built around PyTorch and configurable components. torchtune for explicit PyTorch training recipes
- Train supported language models with configurable datasets and tuning methods. LlamaFactory for configuring model fine-tuning
For the distinction between training from scratch and adapting a checkpoint, see Franklin’s glossary. Fine-tuning
Project documentation
Try it for your workflow
Explore the project’s documentation for setup and usage, or save this page to a collection for later.
Read the project documentation ↗