Metric & Benchmark
Elo Rating
Skill rating computed from pairwise wins — the basis of Chatbot Arena rankings.
Definition
Originally designed for chess, the Elo system updates each player's rating after every match based on result and rating difference. LMSYS Chatbot Arena adapts it to LLMs: users compare two model outputs blindly, and the winning model gains points proportional to upset magnitude.
Common use cases
- Chatbot Arena
- Human-pref ranking
- Public leaderboards