Skip to content

Benchmark

LLM Chess Wrong Moves per 1000 Moves

Lower scores are better.

Models measured
149
Scores recorded
206
Sources
1
IQ weight
1.97

Score distribution

43.7
Mean
9.1
Median
76.9
Std dev
0.483
10th percentile
141
90th percentile
0
Min
0458.3
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.97 for chat models, measured over 149 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Claude Opus 4.5budget_16000· LLM Chess Leaderboard data docs0
2Gemini 3 Pro Previewhigh· LLM Chess Leaderboard data docs0
3Gemini 3.5 Flashmedium· LLM Chess Leaderboard data docs0
4Gemini 3.6 Flashhigh· LLM Chess Leaderboard data docs0
5Gemini 3.6 Flashmedium· LLM Chess Leaderboard data docs0
6Gemini 4 Argonhigh· LLM Chess Leaderboard data docs0
7GPT 4 32knone· LLM Chess Leaderboard data docs0
8GPT-3.5 Turbonone· LLM Chess Leaderboard data docs0
9GPT-5.6 Solxhigh· LLM Chess Leaderboard data docs0
10GPT-6.1 Solxhigh· LLM Chess Leaderboard data docs0
11Muse Emberxhigh· LLM Chess Leaderboard data docs0
12Qwen Maxnone· LLM Chess Leaderboard data docs0
13Grok 4.3high· LLM Chess Leaderboard data docs0.223
14Grok 4.6high· LLM Chess Leaderboard data docs0.287
15OpenAI o3medium· LLM Chess Leaderboard data docs0.301
16GPT-6.1 Solhigh· LLM Chess Leaderboard data docs0.305
17Gemini 3.8 Flashmedium· LLM Chess Leaderboard data docs0.315
18o1low· LLM Chess Leaderboard data docs0.335
19Gemini 3.1 Pro Previewhigh· LLM Chess Leaderboard data docs0.389
20Gemini 3.8 Flashhigh· LLM Chess Leaderboard data docs0.448
21GPT-5.1 Codex Minihigh· LLM Chess Leaderboard data docs0.479
22DeepSeek V4.1 Flash 0910high· LLM Chess Leaderboard data docs0.483
23Gemini 3.7 Flashhigh· LLM Chess Leaderboard data docs0.487
24GPT-5.6 Solhigh· LLM Chess Leaderboard data docs0.534
25GPT 5.1medium· LLM Chess Leaderboard data docs0.655
26Gemini 3.6 Flashlow· LLM Chess Leaderboard data docs0.707
27GPT 5.5medium· LLM Chess Leaderboard data docs0.718
28Muse Glimmer 30Bdefault@q4_k_m· LLM Chess Leaderboard data docs0.718
29Gemini 2.0 Flash Experimentaldefault, thinking· LLM Chess Leaderboard data docs0.746
30GPT-6 Solhigh· LLM Chess Leaderboard data docs0.862
31GPT-6 Astrahigh· LLM Chess Leaderboard data docs0.929
32GPT 5.5high· LLM Chess Leaderboard data docs0.936
33GPT-4 Turbonone· LLM Chess Leaderboard data docs1.2
34Claude Sonnet 4.5budget_16000· LLM Chess Leaderboard data docs1.2
35GPT-5.6 Terraxhigh· LLM Chess Leaderboard data docs1.2
36GPT 5.1low· LLM Chess Leaderboard data docs1.2
37GPT 5medium· LLM Chess Leaderboard data docs1.3
38GPT-6 Solxhigh· LLM Chess Leaderboard data docs1.3
39GPT-5.1 Chatnone· LLM Chess Leaderboard data docs1.4
40Claude 3 Opusnone· LLM Chess Leaderboard data docs1.4
41Claude Opus 4budget_16000· LLM Chess Leaderboard data docs1.5
42GPT-5.6 Solmedium· LLM Chess Leaderboard data docs1.5
43GPT 4.1none· LLM Chess Leaderboard data docs1.6
44GPT 5low· LLM Chess Leaderboard data docs1.7
45GPT-6 Lunaxhigh· LLM Chess Leaderboard data docs1.8
46O1 Minidefault· LLM Chess Leaderboard data docs1.9
47GPT-5.3 Codexmedium· LLM Chess Leaderboard data docs2
48Claude Haiku 4.5budget_16000· LLM Chess Leaderboard data docs2.1
49GPT 4none· LLM Chess Leaderboard data docs2.2
50Grok 4.1 Fastdefault· LLM Chess Leaderboard data docs2.2

Showing 1–50 of 206 results.

Data credits: LLM Chess by Maxim Saplin, Apache-2.0 (Copyright [2024] [Maxim Saplin]) (licence)