Metric & Benchmark
aka TTFT
Time to First Token
Latency between sending a prompt and receiving the model's first response token.
Definition
TTFT measures how long the prefill phase takes before generation begins. It is dominated by prompt length and queueing. For chat apps it is what users feel as responsiveness; TPS controls the streaming experience after that.
Common use cases
- Chat UX
- Streaming APIs
- Serving optimisation