Skip to content

Benchmark

GDM-MRCRv2: 8 needle 512k

Long context benchmarks measure how well models use information in long inputs. Scores are percentages.

Models measured
40
Scores recorded
107
Sources
1
IQ weight
1.96

Score distribution

31.4%
Mean
29.7%
Median
15.4%
Std dev
13.2%
10th percentile
52.5%
90th percentile
7.3%
Min
7.3%77.7%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.96 for chat models, measured over 40 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Gemini 3.7 Flashhigh· Context Arena data77.7%
2Gemini 3.7 Flashmedium· Context Arena data74.4%
3Gemini 3.7 Flashlow· Context Arena data65.1%
4GPT-5.6 Solmax· Context Arena data61.9%
5Gemini 3.6 Flashhigh· Context Arena data61.1%
6GPT 5.5medium· Context Arena data57.6%
7Gemini 3.6 Flashlow· Context Arena data55.4%
8GPT 5.5high· Context Arena data54.4%
9GPT 5.5low· Context Arena data54.3%
10Claude Opus 4.6high· Context Arena data54.3%
11GPT 5.5xhigh· Context Arena data54.2%
12Gemini 3.6 Flashmedium· Context Arena data52.5%
13GPT-5.6 Terramax· Context Arena data49.9%
14Claude Opus 4.6low· Context Arena data49.8%
15Claude Opus 4.6medium· Context Arena data48.9%
16Muse Spark 1.2medium· Context Arena data47.8%
17Muse Spark 1.2low· Context Arena data46.5%
18Claude Sonnet 4.6high· Context Arena data45.9%
19Muse Spark 1.2xhigh· Context Arena data45%
20Muse Spark 1.2high· Context Arena data44.5%
21Claude Sonnet 4.6medium· Context Arena data43.7%
22Claude Sonnet 4.6low· Context Arena data43.5%
23Claude Opus 5max· Context Arena data42.5%
24Muse Spark 1.1minimal· Context Arena data42.5%
25GLM 5.3high· Context Arena data41.9%
26GPT 5.4xhigh· Context Arena data41.5%
27Muse Spark 1.1high· Context Arena data41.3%
28Muse Spark 1.1low· Context Arena data41.3%
29Muse Spark 1.1xhigh· Context Arena data41%
30Muse Spark 1.2minimal· Context Arena data40.5%
31GPT 5.4high· Context Arena data40.5%
32Claude Opus 4.8max· Context Arena data39.8%
33GLM 5.3max· Context Arena data39.3%
34GLM 5.3 Flashhigh· Context Arena data39%
35Gemini 3.5 Flashhigh· Context Arena data38.8%
36GPT 5.4low· Context Arena data38.4%
37GPT 5.4medium· Context Arena data38.3%
38Muse Spark 1.1medium· Context Arena data38%
39GPT-5.6 Lunamax· Context Arena data37.6%
40GLM 5.3low· Context Arena data36.2%
41GLM 5.3 Flashmax· Context Arena data35.4%
42Gemini 3 Flashlow· Context Arena data35%
43GLM 5.2max· Context Arena data35%
44GLM 5.3 Flashlow· Context Arena data34.9%
45DeepSeek V4 Pro 0813max· Context Arena data34.3%
46Gemini 3 Flashhigh· Context Arena data33.6%
47DeepSeek V4 Pro 0813high· Context Arena data33.4%
48Gemini 3 Flashmedium· Context Arena data32.8%
49Claude Sonnet 5max· Context Arena data32%
50Gemini 3.1 Pro Previewhigh· Context Arena data31.1%

Showing 1–50 of 107 results.