Concept
Top-K Sampling
Sampling restricted to the K highest-probability tokens at each step.
Definition
Top-k truncates the model's vocabulary to the k most-likely tokens, renormalises the probabilities and samples from that subset. It avoids low-probability tail words that lead to incoherent text, while still allowing variety beyond pure greedy decoding.
Common use cases
- Controlled creativity
- Story generation
- Dialogue