AI Glossary
Model sharding
What does “Model sharding” mean?
Model sharding: Splitting a model's weights or computation across multiple devices.
What is an example of “Model sharding”?
A model too large for one GPU is loaded across several GPUs.
AI Glossary
Model sharding: Splitting a model's weights or computation across multiple devices.
A model too large for one GPU is loaded across several GPUs.