Concept
Capacity
Informal measure of how much a model can learn, set by parameter count and architecture.
Definition
Capacity refers to a network's ability to memorise and represent patterns, governed mainly by parameter count, depth, width, and architectural priors. Under-capacity models underfit; over-capacity ones risk memorisation without regularisation. Scaling laws relate capacity, data and compute to loss.
Common use cases
- Model selection
- Scaling decisions
- Cost-quality tradeoff