Skip to content

Benchmark

LLM Chess Elo

Scores are Elo ratings. Higher ratings are better.

Models measured
115
Scores recorded
167
Sources
1
IQ weight
1.85

Score distribution

336.4
Mean
115.3
Median
640.5
Std dev
-299.9
10th percentile
1,310.5
90th percentile
-812.4
Min
-812.41,613.8
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.85 for chat models, measured over 115 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
101Claude Sonnet 3.7budget_5000· LLM Chess Leaderboard data docs-36.1
102Grok 3 Mininone, beta· LLM Chess Leaderboard data docs-37.3
103Qwen3 30B A3Bdefault@q4_k_m· LLM Chess Leaderboard data docs-37.3
104Claude Opus 4.1none· LLM Chess Leaderboard data docs-37.9
105Muse Emberxhigh· LLM Chess Leaderboard data docs-38.3
106Claude Haiku 4.5none· LLM Chess Leaderboard data docs-42
107O1 Minidefault· LLM Chess Leaderboard data docs-52.4
108GPT-5 minilow· LLM Chess Leaderboard data docs-55.6
109Claude Sonnet 3.7budget_2048· LLM Chess Leaderboard data docs-62.4
110Gemini 2.5 Pro Preview 05-06default· LLM Chess Leaderboard data docs-70.7
111Claude Sonnet 3.7none· LLM Chess Leaderboard data docs-76.4
112GPT-5 Chatnone· LLM Chess Leaderboard data docs-87.3
113GPT 4 32knone· LLM Chess Leaderboard data docs-97.8
114Claude 3.5 Sonnet v2none· LLM Chess Leaderboard data docs-98.7
115Qwen Maxnone· LLM Chess Leaderboard data docs-98.9
116Llama 4 Mavericknone· LLM Chess Leaderboard data docs-99.7
117GPT-4 Turbonone· LLM Chess Leaderboard data docs-103.8
118GPT 4onone· LLM Chess Leaderboard data docs-108.1
119GPT 4.1none· LLM Chess Leaderboard data docs-110.6
120GPT-4.5 Previewnone· LLM Chess Leaderboard data docs-111
121GPT-5 nanolow· LLM Chess Leaderboard data docs-117
122Grok 4.1 Fastnone· LLM Chess Leaderboard data docs-118.4
123GPT 4none· LLM Chess Leaderboard data docs-119.8
124Claude Haiku 3.5none· LLM Chess Leaderboard data docs-120.5
125Grok 4.20none· LLM Chess Leaderboard data docs-125
126Qwen3 Next 80B A3Bnone· LLM Chess Leaderboard data docs-127.9
127GPT-5.1 Chatnone· LLM Chess Leaderboard data docs-130.7
128Gemini 3.1 Flash Litedefault· LLM Chess Leaderboard data docs-132.9
129Claude 3.5 Sonnetnone· LLM Chess Leaderboard data docs-134.4
130Grok 3none, beta· LLM Chess Leaderboard data docs-137.2
131Qwen3 235B A22B Instruct 2507none· LLM Chess Leaderboard data docs-140.3
132Gemini 2.5 Pro Preview 03-25default· LLM Chess Leaderboard data docs-144.3
133Claude 3 Opusnone· LLM Chess Leaderboard data docs-145.7
134Gemini 2.5 Prodefault· LLM Chess Leaderboard data docs-147.2
135GPT-4.1 mininone· LLM Chess Leaderboard data docs-167.2
136Gemini 3 Flashdefault· LLM Chess Leaderboard data docs-179.5
137o3 Minilow· LLM Chess Leaderboard data docs-183.9
138Grok 4 Fastnone· LLM Chess Leaderboard data docs-191.3
139Qwen3.5 9Bdefault@bf16· LLM Chess Leaderboard data docs-199.6
140GPT OSS 120Bmedium· LLM Chess Leaderboard data docs-217.2
141GPT OSS 20bmedium· LLM Chess Leaderboard data docs-232
142GPT-4o mininone· LLM Chess Leaderboard data docs-251.9
143GPT OSS 120Bhigh· LLM Chess Leaderboard data docs-252.1
144GPT OSS 20bhigh· LLM Chess Leaderboard data docs-257.7
145Qwen3.5 9Bnone@q8_0· LLM Chess Leaderboard data docs-257.7
146Llama 3 70B Instructnone· LLM Chess Leaderboard data docs-278.1
147GPT OSS 120Blow· LLM Chess Leaderboard data docs-283.6
148Gemini 2.5 Flashdefault· LLM Chess Leaderboard data docs-289.3
149GLM 5default· LLM Chess Leaderboard data docs-299.2
150GPT OSS 20blow· LLM Chess Leaderboard data docs-299.9

Showing 101–150 of 167 results.

Data credits: LLM Chess by Maxim Saplin, Apache-2.0 (Copyright [2024] [Maxim Saplin]) (licence)