Skip to content

Benchmark

GDM-MRCRv2: 8 needle 512k

Long context benchmarks measure how well models use information in long inputs. Scores are percentages.

Models measured
40
Scores recorded
107
Sources
1
IQ weight
1.96

Score distribution

31.4%
Mean
29.7%
Median
15.4%
Std dev
13.2%
10th percentile
52.5%
90th percentile
7.3%
Min
7.3%77.7%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.96 for chat models, measured over 40 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
51Gemini 3.5 Flashmedium· Context Arena data30.8%
52DeepSeek V4 Flash 0731max· Context Arena data29.8%
53Claude Sonnet 4.6· Context Arena data29.8%
54Qwen3.8 Maxxhigh· Context Arena data29.7%
55Gemini 3.6 Flashminimal· Context Arena data29.1%
56DeepSeek V4 Proenabled· Context Arena data28.8%
57Gemini 3.5 Flash Litemedium· Context Arena data28.5%
58Qwen3.8 Maxlow· Context Arena data28.5%
59DeepSeek V4 Pro 0813low· Context Arena data28.3%
60Claude Opus 4.6· Context Arena data28.2%
61DeepSeek V4 Flash 0731high· Context Arena data28.1%
62Gemini 3.1 Pro Previewlow· Context Arena data27.9%
63Gemini 3.5 Flash Litehigh· Context Arena data26.8%
64Qwen3.6 Plus· Context Arena data26.3%
65Qwen3.8 Maxmedium· Context Arena data25.7%
66Gemini 3.5 Flash Litelow· Context Arena data25.6%
67Kimi K3max· Context Arena data25.3%
68GPT 5.5· Context Arena data24.9%
69DeepSeek V4 Flashenabled· Context Arena data24.7%
70DeepSeek V4 Flash 0731low· Context Arena data24.1%
71Qwen3.7 Flash· Context Arena data23.7%
72Qwen3.7 Max· Context Arena data22.8%
73Gemini 3.5 Flashlow· Context Arena data21.4%
74Qwen3.7 Flashenabled· Context Arena data21.3%
75Gemini 3 Flash· Context Arena data20.6%
76DeepSeek V4 Pro 0813· Context Arena data20.6%
77Qwen3.6 Flash· Context Arena data20.4%
78Gemini 3.5 Flash Liteminimal· Context Arena data19.7%
79Kimi K3low· Context Arena data19.5%
80Qwen3.6 Plusenabled· Context Arena data19.3%
81Kimi K3high· Context Arena data18.9%
82Gemini 3.1 Flash Litelow· Context Arena data18.6%
83DeepSeek V4 Pro· Context Arena data18.1%
84DeepSeek V4 Flash· Context Arena data18%
85GPT 5.4· Context Arena data17.7%
86Gemini 3.1 Flash Litemedium· Context Arena data17.1%
87Gemini 3.1 Flash Litehigh· Context Arena data16.9%
88Gemini 3.5 Flash· Context Arena data15.9%
89DeepSeek V4 Flash 0731· Context Arena data15%
90Qwen3.5 Plus· Context Arena data14.7%
91Gemini 3.1 Flash Lite· Context Arena data14%
92MiMo V2.5 Pro· Context Arena data13.9%
93MiMo V2.5· Context Arena data13.6%
94Qwen3.5 Flash· Context Arena data13.3%
95MiMo V2.5enabled· Context Arena data13.2%
96Qwen3.5 Plusenabled· Context Arena data13.2%
97MiMo V2.5 Proenabled· Context Arena data12.9%
98Grok 4.20· Context Arena data11.8%
99Grok 4.20enabled· Context Arena data11.7%
100MiMo V2 Proenabled· Context Arena data9.8%

Showing 51–100 of 107 results.