Skip to content

Benchmark

LLM Chess Wrong Moves per 1000 Moves

Lower scores are better.

Models measured
149
Scores recorded
206
Sources
1
IQ weight
1.97

Score distribution

43.7
Mean
9.1
Median
76.9
Std dev
0.483
10th percentile
141
90th percentile
0
Min
0458.3
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.97 for chat models, measured over 149 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
51GPT-5.1 Codexlow· LLM Chess Leaderboard data docs2.3
52OpenAI o3low· LLM Chess Leaderboard data docs2.4
53Claude Opus 4.5none· LLM Chess Leaderboard data docs2.5
54O1 Previewdefault· LLM Chess Leaderboard data docs2.7
55Claude Sonnet 4budget_16000· LLM Chess Leaderboard data docs2.7
56GPT-6 Lunahigh· LLM Chess Leaderboard data docs2.7
57Claude Sonnet 3.7budget_10000· LLM Chess Leaderboard data docs2.8
58Claude Sonnet 3.7budget_5000· LLM Chess Leaderboard data docs2.8
59GPT-5.6 Terramedium· LLM Chess Leaderboard data docs3
60Claude 3 Haikunone· LLM Chess Leaderboard data docs3
61GPT-5.6 Terrahigh· LLM Chess Leaderboard data docs3
62GPT-6 Astraxhigh· LLM Chess Leaderboard data docs3.1
63Grok 4.20default· LLM Chess Leaderboard data docs3.2
64o1medium· LLM Chess Leaderboard data docs3.7
65Gemini 3 Flashdefault· LLM Chess Leaderboard data docs3.8
66Claude Sonnet 3.7budget_2048· LLM Chess Leaderboard data docs3.9
67GPT-6.1 Terraxhigh· LLM Chess Leaderboard data docs4
68Claude Opus 5high· LLM Chess Leaderboard data docs4
69GPT 4onone· LLM Chess Leaderboard data docs4
70GPT 5.4medium· LLM Chess Leaderboard data docs4.2
71Qwen3 Next 80B A3Bnone· LLM Chess Leaderboard data docs4.2
72Claude Sonnet 5.5high· LLM Chess Leaderboard data docs4.3
73GPT 5.4high· LLM Chess Leaderboard data docs4.4
74o4 Minihigh· LLM Chess Leaderboard data docs4.6
75Grok 3 Mininone, beta· LLM Chess Leaderboard data docs4.6
76GPT-5 minilow· LLM Chess Leaderboard data docs4.8
77o4 Minimedium· LLM Chess Leaderboard data docs4.8
78Grok 4.7high· LLM Chess Leaderboard data docs4.8
79Grok 3 Minilow, beta· LLM Chess Leaderboard data docs4.8
80Claude 3.5 Sonnetnone· LLM Chess Leaderboard data docs4.9
81Llama 4 Mavericknone· LLM Chess Leaderboard data docs4.9
82GPT-4.5 Previewnone· LLM Chess Leaderboard data docs5
83Gemini 3.1 Flash Litedefault· LLM Chess Leaderboard data docs5.1
84GPT-6.1 Terrahigh· LLM Chess Leaderboard data docs5.1
85Claude Opus 4.6none· LLM Chess Leaderboard data docs5.3
86Grok 4 Fastdefault· LLM Chess Leaderboard data docs5.4
87GPT-4.1 mininone· LLM Chess Leaderboard data docs5.5
88GPT-5.6 Lunahigh· LLM Chess Leaderboard data docs5.7
89GPT-5.6 Lunaxhigh· LLM Chess Leaderboard data docs6
90Mercury 3high· LLM Chess Leaderboard data docs6.1
91GPT 5.2medium· LLM Chess Leaderboard data docs6.1
92o4 Minilow· LLM Chess Leaderboard data docs6.2
93GPT-5 minihigh· LLM Chess Leaderboard data docs6.3
94Claude Opus 5.5high· LLM Chess Leaderboard data docs6.4
95GPT 5.2low· LLM Chess Leaderboard data docs6.6
96Claude Sonnet 4.6none· LLM Chess Leaderboard data docs6.7
97Claude 3.5 Sonnet v2none· LLM Chess Leaderboard data docs6.8
98Claude Sonnet 4.6high· LLM Chess Leaderboard data docs7
99MiniMax M2.5none· LLM Chess Leaderboard data docs7.1
100Claude Sonnet 3.7budget_1024· LLM Chess Leaderboard data docs7.2

Showing 51–100 of 206 results.

Data credits: LLM Chess by Maxim Saplin, Apache-2.0 (Copyright [2024] [Maxim Saplin]) (licence)