AI Glossary

Hyperparameter

What does “Hyperparameter” mean?

Hyperparameter: A setting chosen by people before or during training, rather than learned from data, such as the learning rate, batch size, number of epochs or model size. Finding good values usually takes experiments, and poor choices can waste an entire training run.

Hyperparameter visual guide: A setting chosen by people before or during training, rather than learned from data, such as the learning rate, batch size, number of epochs or model size. Finding good values usually takes experiments, and poor choices can waste an entire training run.
Visual guide Open full size (new tab)

What is an example of “Hyperparameter”?

Teams often try several learning rates on a small model first, then carry the best value over to a much larger run.

Keep learning

See also

Learning rateTermBatch sizeTermEpochTermParametersTerm

Same topic

Instruction tuningTermGradient descentTermKnowledge distillationTermFine-tuningTermLoRATerm