Skip to content

Benchmark

GDM-MRCRv2: 8 needle 1M

Long context benchmarks measure how well models use information in long inputs. Scores are percentages.

Models measured
8
Scores recorded
27
Sources
1
IQ weight
1.27

Score distribution

27.3%
Mean
22%
Median
17.5%
Std dev
10.6%
10th percentile
58.3%
90th percentile
8.2%
Min
8.2%63.5%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.27 for chat models, measured over 8 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard