Metric & Benchmark
Perplexity
Exponential of the average per-token loss; lower means the model finds the data less surprising.
Definition
Perplexity measures how well a language model predicts a held-out sample. Mathematically it is exp(cross-entropy). It is meaningful only when comparing models with the same tokeniser on the same text; absolute values across vocabularies are not directly comparable.
Common use cases
- Pre-training monitoring
- Tokenizer comparison
- Domain fit