Metric & Benchmark
aka TPS
Tokens per Second
Per-stream generation speed of a model — the most reported speed number for LLMs.
Definition
Tokens per second is the steady-state speed at which a single user's stream emits tokens. It excludes the time-to-first-token (TTFT). Modern frontier models on H100/B200 GPUs hit 50–200 tok/s; smaller models on consumer hardware approach 100 tok/s with llama.cpp.
Common use cases
- UX benchmarking
- Hardware comparison