Concept
aka model size
Parameter Count
Number of learned weights in a neural network — a rough proxy for capacity and cost.
Definition
Parameter count, usually reported in millions or billions, summarises a model's raw size. More parameters generally mean better quality and higher memory and compute cost; scaling laws relate parameter count, dataset size, and compute. Mixture-of-experts complicates the picture because only a subset is active per token.
Common use cases
- Capacity planning
- Cost estimation
- Benchmark comparison