Skip to content

Benchmark

Position Bias (Lech Mazur)

Scores are percentages. Lower is better.

Models measured
30
Scores recorded
35
Sources
1
IQ weight
1.02

Score distribution

43.4%
Mean
43.6%
Median
12.2%
Std dev
29.8%
10th percentile
58.7%
90th percentile
19.8%
Min
19.8%72.5%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.02 for chat models, measured over 30 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1MiMo V2 Pro· Lech Mazur suite data19.8%
2Claude Fable 5medium reasoning· Lech Mazur suite data27.3%
3Claude Opus 4.8high reasoning· Lech Mazur suite data27.4%
4Gemini 3.5 Flash· Lech Mazur suite data29.8%
5Claude Opus 4.6high reasoning· Lech Mazur suite data30.2%
6DeepSeek V3.2· Lech Mazur suite data30.3%
7GLM 5.1· Lech Mazur suite data31.5%
8Qwen3.6 Plus· Lech Mazur suite data33.8%
9Qwen3.7 Max· Lech Mazur suite data34.8%
10MiniMax M3· Lech Mazur suite data34.9%
11Gemini 3.1 Pro Preview· Lech Mazur suite data35.4%
12MiniMax M2.7· Lech Mazur suite data36.5%
13Trinity Large Thinking· Lech Mazur suite data36.6%
14Claude Sonnet 4.6high reasoning· Lech Mazur suite data37.4%
15Claude Opus 4.7high reasoning· Lech Mazur suite data37.7%
16Claude Opus 4.6no reasoning· Lech Mazur suite data38.1%
17Grok 4.20Reasoning· Lech Mazur suite data39%
18DeepSeek V4 Pro· Lech Mazur suite data43.6%
19GPT 5.5medium reasoning· Lech Mazur suite data44.4%
20Claude Sonnet 4.6no reasoning· Lech Mazur suite data44.8%
21ERNIE 5.0· Lech Mazur suite data45%
22Gemini 3.1 Flash Lite· Lech Mazur suite data45.3%
23Kimi K2.6· Lech Mazur suite data47.3%
24Qwen3.5 397B A17B· Lech Mazur suite data48.5%
25Gemma 4 31B IT· Lech Mazur suite data50.8%
26Kimi K2.5thinking· Lech Mazur suite data51.6%
27Qwen3.5 122B-A10B· Lech Mazur suite data51.7%
28GPT-5.4 minixhigh reasoning· Lech Mazur suite data54.7%
29Gemma 4 31B ITreasoning· Lech Mazur suite data54.9%
30GPT 5.4no reasoning· Lech Mazur suite data57.4%
31Mistral Medium 3.1· Lech Mazur suite data57.9%
32Mistral Large 3· Lech Mazur suite data58.7%
33Grok 4.20Non-Reasoning· Lech Mazur suite data64.8%
34GPT 5.4high reasoning· Lech Mazur suite data66.3%
35Mistral Medium 3.5· Lech Mazur suite data72.5%