AI GitHub
ThunderKittens for writing GPU kernels used in AI
Study GPU kernel building blocks and examples for tensor-heavy workloads.
Key Takeaways
- The library supplies primitives and examples for writing kernels around modern GPU operations.
- Check the example's hardware requirements and compiler setup.
- Build a documented example on supported hardware and compare its output with a reference.
- A fast kernel for one size or architecture can perform differently elsewhere.
How it works
The library supplies primitives and examples for writing kernels around modern GPU operations. It is most relevant when profiling shows an existing operation dominates runtime.
What you need
Check the example's hardware requirements and compiler setup. Use a fixed matrix size and reference output before exploring kernel tiling or other tuning choices.
What to try first
Build a documented example on supported hardware and compare its output with a reference. Measure steady-state performance across several problem sizes.
Measure against an existing implementation with the same precision. Compare output tolerance before treating an apparent speedup as a valid substitute.
Where to look for failures
A fast kernel for one size or architecture can perform differently elsewhere. Test boundaries, precision and device compatibility before reusing it.
Related Franklin pages
Profile a real workload before swapping an operation. Verify output and gradient tolerance and compare performance over the shapes the application uses.
- Build and train neural networks with tensors, automatic differentiation and modules. PyTorch for tensor programs and neural-network training
- Use Triton kernels for supported training operations and integrations. Liger Kernel for studying efficient language-model training operations
For the role of multilayer networks and how they learn, see Franklin’s glossary. Deep learning
Project documentation
Try it for your workflow
Explore the project’s documentation for setup and usage, or save this page to a collection for later.
Read the project documentation ↗