Skip to content

Benchmark

Elimination Game (Lech Mazur)

Scores are percentages.

Models measured
53
Scores recorded
56
Sources
1
IQ weight
1.73

Score distribution

4%
Mean
4.1%
Median
1.2%
Std dev
2.7%
10th percentile
5.5%
90th percentile
0.296%
Min
0.296%7.5%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.73 for chat models, measured over 53 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1GPT 5.2medium reasoning· Lech Mazur suite data7.5%
2GPT 5medium reasoning· Lech Mazur suite data6%
3GPT-5 minimedium reasoning· Lech Mazur suite data5.7%
4Claude Opus 4.5thinking, 16k· Lech Mazur suite data5.7%
5Gemini 3 Flash· Lech Mazur suite data5.7%
6Grok 3 Minihigh reasoning, beta· Lech Mazur suite data5.5%
7GPT 4o· Lech Mazur suite data5.5%
8Claude Sonnet 3.7thinking, 16k· Lech Mazur suite data5.3%
9Claude Opus 4.1no reasoning· Lech Mazur suite data5.2%
10Claude Sonnet 4.5thinking, 16k· Lech Mazur suite data5.2%
11Grok 4· Lech Mazur suite data5.2%
12GPT-4.5 Preview· Lech Mazur suite data5.1%
13Claude 3.5 Sonnet· Lech Mazur suite data5.1%
14Grok 3no reasoning, beta· Lech Mazur suite data5.1%
15Gemini 3 Pro Preview· Lech Mazur suite data4.9%
16Claude Sonnet 3.7· Lech Mazur suite data4.8%
17Gemini 2.5 Flash· Lech Mazur suite data4.7%
18Claude Sonnet 4no reasoning· Lech Mazur suite data4.6%
19MiniMax M2· Lech Mazur suite data4.6%
20Qwen3 Max· Lech Mazur suite data4.5%
21OpenAI o3medium reasoning· Lech Mazur suite data4.5%
22Gemini 2.5 Pro· Lech Mazur suite data4.5%
23Claude Opus 4no reasoning· Lech Mazur suite data4.4%
24o3 Minimedium reasoning· Lech Mazur suite data4.4%
25Kimi K2 Thinking64k· Lech Mazur suite data4.3%
26Claude Sonnet 4thinking, 16k· Lech Mazur suite data4.3%
27GLM 4.5· Lech Mazur suite data4.2%
28Mistral Large 2· Lech Mazur suite data4.1%
29DeepSeek V3· Lech Mazur suite data4.1%
30DeepSeek R1· Lech Mazur suite data4.1%
31Claude Opus 4thinking, 16k· Lech Mazur suite data3.9%
32o1medium reasoning· Lech Mazur suite data3.9%
33GPT OSS 120B· Lech Mazur suite data3.8%
34Mistral Large 3· Lech Mazur suite data3.6%
35Llama 4 Maverick· Lech Mazur suite data3.6%
36Grok 4.1 Fastreasoning· Lech Mazur suite data3.6%
37Llama 3.3 70B Instruct· Lech Mazur suite data3.6%
38Nova Pro· Lech Mazur suite data3.5%
39Qwen 3 235B A22B· Lech Mazur suite data3.5%
40MiniMax Text 01· Lech Mazur suite data3.5%
41Kimi K2· Lech Mazur suite data3.4%
42Mistral-Small-3· Lech Mazur suite data3.4%
43GPT-4o mini· Lech Mazur suite data3.3%
44o4 Minihigh reasoning· Lech Mazur suite data3.3%
45Claude Haiku 3.5· Lech Mazur suite data3.2%
46gemini-2.0-pro-exp-02-05· Lech Mazur suite data3.1%
47Llama 3.1 405B· Lech Mazur suite data3.1%
48Phi 4· Lech Mazur suite data2.8%
49Gemini 2.0 Flash Experimentalthink· Lech Mazur suite data2.8%
50GLM 4.6· Lech Mazur suite data2.7%

Showing 1–50 of 56 results.