Concept
Sampling
Decoding strategy that picks the next token from the model's output distribution.
Definition
Sampling controls how a model converts logits to tokens. Greedy decoding picks the argmax; temperature, top-k and top-p inject controlled randomness; beam search explores multiple paths. Different strategies trade off determinism, diversity, and quality.
Common use cases
- Creative writing
- Reproducible tests
- Diverse generations