AI Glossary

QLoRA

Also called quantized LoRA

What does “QLoRA” mean?

QLoRA: Combines LoRA with quantization. The frozen base model is stored in low precision, typically 4-bit, while the small adapters are trained at higher precision. This cuts memory use enough to fine-tune fairly large models on a single GPU.

QLoRA visual guide: Combines LoRA with quantization. The frozen base model is stored in low precision, typically 4-bit, while the small adapters are trained at higher precision. This cuts memory use enough to fine-tune fairly large models on a single GPU.
Visual guide Open full size (new tab)

What is an example of “QLoRA”?

QLoRA made it practical for individual researchers and hobbyists to fine-tune open-weight models without renting a large GPU cluster.

Keep learning

See also

LoRATermQuantizationTermFine-tuningTerm

Same topic

RegularizationTermPre-trainingTermReinforcement learning from AI feedbackTermOverfittingTermReinforcement learning from human feedbackTerm