Skip to content

Benchmark

GDM-MRCRv2: 8 needle

Long context benchmarks measure how well models use information in long inputs. Scores are percentages.

Models measured
68
Scores recorded
177
Sources
1
IQ weight
1.21

Score distribution

49.7%
Mean
50.1%
Median
20.2%
Std dev
24%
10th percentile
73.7%
90th percentile
12.3%
Min
12.3%94%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.21 for chat models, measured over 68 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
101DeepSeek V4 Flash 0731low· Context Arena data45.7%
102GPT-5.4 minixhigh· Context Arena data45.7%
103Gemini 3.5 Flash Litehigh· Context Arena data45.2%
104Qwen3.7 Max· Context Arena data45%
105GPT-5.4 minihigh· Context Arena data44.8%
106MiniMax M3enabled· Context Arena data43.8%
107Gemini 3.5 Flash Litemedium· Context Arena data43.6%
108Qwen3.5 9Benabled· Context Arena data43.3%
109Qwen3.6 Flash· Context Arena data42.4%
110GPT-5.4 minimedium· Context Arena data42.1%
111Kimi K3low· Context Arena data41.6%
112Gemini 3.5 Flash Litelow· Context Arena data40.9%
113Gemini 3 Flashlow· Context Arena data39.6%
114MiMo V2 Omnienabled· Context Arena data37.9%
115Step 3.7 Flashmedium· Context Arena data37.7%
116Qwen3.5 27B· Context Arena data36.8%
117Claude Haiku 4.5enabled· Context Arena data36.3%
118MiMo V2.5 Proenabled· Context Arena data36.2%
119GPT 5.4 Nanohigh· Context Arena data36.2%
120Step 3.7 Flashhigh· Context Arena data35.3%
121Qwen3.5 397B A17B· Context Arena data35%
122Gemini 3.1 Flash Litehigh· Context Arena data34.9%
123GPT 5.5· Context Arena data34.9%
124Doubao Seed 2.1 Turbo· Context Arena data34.9%
125GPT-5.4 minilow· Context Arena data34.5%
126Qwen3.5 122B-A10B· Context Arena data34.5%
127Step 3.7 Flashlow· Context Arena data34%
128Gemini 3.5 Flashlow· Context Arena data33.4%
129MiniMax M2.7enabled· Context Arena data33.3%
130Qwen3.5 35B A3B· Context Arena data33.1%
131Mistral Medium 3.5high· Context Arena data32%
132Gemini 3.1 Flash Litemedium· Context Arena data31.8%
133MiMo V2.5enabled· Context Arena data31.6%
134Gemini 3.6 Flashminimal· Context Arena data30.5%
135GLM 5.1· Context Arena data30.3%
136DeepSeek V4 Pro 0813· Context Arena data30.1%
137MiMo V2 Proenabled· Context Arena data30%
138GPT 5.4 Nanomedium· Context Arena data29.9%
139Nemotron 3 Super 120B A12B· Context Arena data29.3%
140Inkling Smallmax· Context Arena data29.2%
141Claude Opus 4.7medium· Context Arena data28.8%
142Grok 4.20enabled· Context Arena data28.8%
143Claude Opus 4.7low· Context Arena data28.6%
144Claude Opus 4.7xhigh· Context Arena data28.5%
145Claude Opus 4.7high· Context Arena data28%
146Qwen3.5 Plus· Context Arena data27.9%
147Qwen3.5 Flash· Context Arena data27.2%
148Gemini 3.1 Flash Litelow· Context Arena data26.9%
149Inkling Smallhigh· Context Arena data26.8%
150GPT 5.4· Context Arena data26.7%

Showing 101–150 of 177 results.