Concept
aka objective
Loss Function
Scalar measure of how wrong a model's predictions are; gradient descent minimises it.
Definition
The loss function defines the optimisation target during training. Cross-entropy is standard for language modelling; mean-squared error for regression; contrastive losses for representation learning. The loss landscape's geometry — its smoothness, sharp minima — strongly affects training stability and generalisation.
Common use cases
- Training
- Evaluation
- Loss curve diagnostics