AI GitHub
Liger Kernel for studying efficient language-model training operations
Use Triton kernels for supported training operations and integrations.
Key Takeaways
- Liger replaces selected model operations with kernels intended to reduce memory traffic or intermediate allocations.
- Choose a supported integration and identify the operations it replaces.
- Compare one supported operation against the original implementation for forward output and gradients.
- Kernel support depends on accelerator, dtype and library versions.
How it works
Liger replaces selected model operations with kernels intended to reduce memory traffic or intermediate allocations. The benefit depends on the operation mix and model integration.
What you need
Choose a supported integration and identify the operations it replaces. Keep a small numerical reference before enabling the kernels in a full training job.
What to try first
Compare one supported operation against the original implementation for forward output and gradients. Measure memory and speed with representative shapes.
Profile the original training step before enabling the replacement. A faster operation matters more if it dominates the workload than if data loading is the bottleneck.
Where to look for failures
Kernel support depends on accelerator, dtype and library versions. Verify numerical tolerance and fallback behavior before enabling it for a full training run.
Related Franklin pages
Profile a real workload before swapping an operation. Verify output and gradient tolerance and compare performance over the shapes the application uses.
- Study GPU kernel building blocks and examples for tensor-heavy workloads. ThunderKittens for writing GPU kernels used in AI
- Build and train neural networks with tensors, automatic differentiation and modules. PyTorch for tensor programs and neural-network training
For the role of multilayer networks and how they learn, see Franklin’s glossary. Deep learning
Project documentation
Try it for your workflow
Explore the project’s documentation for setup and usage, or save this page to a collection for later.
Read the project documentation ↗