Skip to content

Benchmark

LLM Chess Elo

Scores are Elo ratings. Higher ratings are better.

Models measured
115
Scores recorded
167
Sources
1
IQ weight
1.85

Score distribution

336.4
Mean
115.3
Median
640.5
Std dev
-299.9
10th percentile
1,310.5
90th percentile
-812.4
Min
-812.41,613.8
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.85 for chat models, measured over 115 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
51GPT-6 Lunahigh· LLM Chess Leaderboard data docs730.3
52GPT-5.6 Lunamedium· LLM Chess Leaderboard data docs730.1
53GPT-5 nanohigh· LLM Chess Leaderboard data docs683
54GPT 5.1low· LLM Chess Leaderboard data docs614.7
55GLM 5.3high· LLM Chess Leaderboard data docs605.2
56GLM 5.3 Flashhigh· LLM Chess Leaderboard data docs571.7
57GPT-5.1 Codex Minihigh· LLM Chess Leaderboard data docs544
58GPT-5.1 Codexlow· LLM Chess Leaderboard data docs504.3
59Gemini 3.6 Flashlow· LLM Chess Leaderboard data docs498.4
60Claude Opus 4.5budget_16000· LLM Chess Leaderboard data docs446.1
61Grok 4.6high· LLM Chess Leaderboard data docs442.7
62o4 Minihigh· LLM Chess Leaderboard data docs440
63Grok 3 Minihigh, beta· LLM Chess Leaderboard data docs430.4
64GPT-5.4 minihigh· LLM Chess Leaderboard data docs416.4
65GPT-5 nanomedium· LLM Chess Leaderboard data docs413.3
66o3 Minihigh· LLM Chess Leaderboard data docs403.1
67GPT-5 minimedium· LLM Chess Leaderboard data docs348.5
68GPT 5.4low· LLM Chess Leaderboard data docs347.9
69Agents A1default@q4_k_m· LLM Chess Leaderboard data docs303.8
70DeepSeek R1default· LLM Chess Leaderboard data docs299.7
71o1medium· LLM Chess Leaderboard data docs280.1
72Claude Sonnet 5none· LLM Chess Leaderboard data docs272.2
73Qwen3.6 27Bdefault@q4_k_s· LLM Chess Leaderboard data docs270
74o4 Minimedium· LLM Chess Leaderboard data docs240.3
75Muse Glimmer 30Bdefault@q4_k_m· LLM Chess Leaderboard data docs214.4
76GPT 5.4 Nanohigh· LLM Chess Leaderboard data docs207.3
77Claude Sonnet 4.6high· LLM Chess Leaderboard data docs189.7
78DeepSeek V3.2none· LLM Chess Leaderboard data docs189.2
79Grok 4.3high· LLM Chess Leaderboard data docs178.6
80GPT-5.4 minilow· LLM Chess Leaderboard data docs176.2
81o1low· LLM Chess Leaderboard data docs153.7
82GPT-5.2 Chatnone· LLM Chess Leaderboard data docs142.4
83Claude Haiku 4.5budget_16000· LLM Chess Leaderboard data docs126.6
84Claude Sonnet 4budget_16000· LLM Chess Leaderboard data docs115.3
85Claude Sonnet 4.5budget_16000· LLM Chess Leaderboard data docs112.7
86Kimi K2.5default· LLM Chess Leaderboard data docs111.2
87Claude Sonnet 4.6none· LLM Chess Leaderboard data docs110.6
88o4 Minilow@PGN· LLM Chess Leaderboard data docs109.9
89o4 Minilow· LLM Chess Leaderboard data docs104.1
90o3 Minimedium· LLM Chess Leaderboard data docs82.4
91Claude Opus 4.5none· LLM Chess Leaderboard data docs71.1
92O1 Previewdefault· LLM Chess Leaderboard data docs46.3
93Claude Opus 4budget_16000· LLM Chess Leaderboard data docs29.3
94Claude Sonnet 3.7budget_10000· LLM Chess Leaderboard data docs-1
95Claude Opus 4none· LLM Chess Leaderboard data docs-10.9
96Claude Sonnet 4none· LLM Chess Leaderboard data docs-10.9
97Qwen3.5 35B A3Bdefault@q4_k_m· LLM Chess Leaderboard data docs-26
98Grok 3 Minilow, beta· LLM Chess Leaderboard data docs-33.9
99Claude Sonnet 4.5none· LLM Chess Leaderboard data docs-35.9
100Claude Sonnet 3.7budget_1024· LLM Chess Leaderboard data docs-36.1

Showing 51–100 of 167 results.

Data credits: LLM Chess by Maxim Saplin, Apache-2.0 (Copyright [2024] [Maxim Saplin]) (licence)