Skip to content

Benchmark

LLM Chess Elo

Scores are Elo ratings. Higher ratings are better.

Models measured
115
Scores recorded
167
Sources
1
IQ weight
1.85

Score distribution

336.4
Mean
115.3
Median
640.5
Std dev
-299.9
10th percentile
1,310.5
90th percentile
-812.4
Min
-812.41,613.8
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.85 for chat models, measured over 115 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1GPT-6 Astrahigh· LLM Chess Leaderboard data docs1,613.8
2GPT-6.1 Solxhigh· LLM Chess Leaderboard data docs1,611.6
3Gemini 4 Argonhigh· LLM Chess Leaderboard data docs1,609.4
4GPT-6 Astraxhigh· LLM Chess Leaderboard data docs1,579.3
5Claude Opus 5.5high· LLM Chess Leaderboard data docs1,575.3
6GPT-5.6 Solxhigh· LLM Chess Leaderboard data docs1,549.7
7GPT-5.6 Solhigh· LLM Chess Leaderboard data docs1,548.5
8Gemini 3.8 Flashhigh· LLM Chess Leaderboard data docs1,546.1
9GPT-6 Solxhigh· LLM Chess Leaderboard data docs1,545.4
10GPT 5.5medium· LLM Chess Leaderboard data docs1,532.2
11GPT 5.5high· LLM Chess Leaderboard data docs1,528.4
12Gemini 3.8 Flashmedium· LLM Chess Leaderboard data docs1,526.5
13GPT-6.1 Terraxhigh· LLM Chess Leaderboard data docs1,521.2
14Gemini 3.1 Pro Previewhigh· LLM Chess Leaderboard data docs1,511.4
15GPT-6.1 Solhigh· LLM Chess Leaderboard data docs1,454.4
16GPT-6.1 Terrahigh· LLM Chess Leaderboard data docs1,391.1
17GPT 5.4high· LLM Chess Leaderboard data docs1,322.3
18GPT-6 Solhigh· LLM Chess Leaderboard data docs1,310.5
19Mercury 3high· LLM Chess Leaderboard data docs1,308
20GPT-5.6 Terraxhigh· LLM Chess Leaderboard data docs1,293
21Claude Opus 5high· LLM Chess Leaderboard data docs1,285.4
22GPT 5.4medium· LLM Chess Leaderboard data docs1,252.9
23GPT-5.6 Solmedium· LLM Chess Leaderboard data docs1,240
24GPT-5.6 Terrahigh· LLM Chess Leaderboard data docs1,210.7
25Claude Sonnet 5.5high· LLM Chess Leaderboard data docs1,194.6
26GPT-5.6 Lunaxhigh· LLM Chess Leaderboard data docs1,185.3
27Claude Haiku 5.5high· LLM Chess Leaderboard data docs1,142.8
28Gemini 3.7 Flashhigh· LLM Chess Leaderboard data docs1,107.1
29Gemini 3.6 Flashhigh· LLM Chess Leaderboard data docs1,093.7
30DeepSeek V4.1 Flash 0910high· LLM Chess Leaderboard data docs1,086.9
31GPT 5medium· LLM Chess Leaderboard data docs1,086.4
32GPT 5.2medium· LLM Chess Leaderboard data docs1,085.1
33GPT-5.3 Codexmedium· LLM Chess Leaderboard data docs1,053.1
34Gemini 3 Pro Previewhigh· LLM Chess Leaderboard data docs1,033.7
35Gemini 3.5 Flashmedium· LLM Chess Leaderboard data docs952.9
36GPT 5.1medium· LLM Chess Leaderboard data docs947.6
37GPT-5.6 Terramedium· LLM Chess Leaderboard data docs935.9
38Gemini 3.6 Flashmedium· LLM Chess Leaderboard data docs921.3
39GPT-5.6 Lunahigh· LLM Chess Leaderboard data docs919.4
40GPT 5low· LLM Chess Leaderboard data docs858.2
41GPT-6 Lunaxhigh· LLM Chess Leaderboard data docs855
42Qwen 3.8 27Bdefault@q4_k_m· LLM Chess Leaderboard data docs845.6
43Grok 4.20default· LLM Chess Leaderboard data docs843
44Grok 4.7high· LLM Chess Leaderboard data docs822.2
45Grok 4.1 Fastdefault· LLM Chess Leaderboard data docs799.5
46GPT 5.2low· LLM Chess Leaderboard data docs782.5
47OpenAI o3medium· LLM Chess Leaderboard data docs777.6
48GPT-5 minihigh· LLM Chess Leaderboard data docs759
49OpenAI o3low· LLM Chess Leaderboard data docs738.6
50Grok 4 Fastdefault· LLM Chess Leaderboard data docs735.1

Showing 1–50 of 167 results.

Data credits: LLM Chess by Maxim Saplin, Apache-2.0 (Copyright [2024] [Maxim Saplin]) (licence)