Skip to content

Benchmark

OpenAI-MRCR: 2 needle 1M

Long context benchmarks measure how well models use information in long inputs. Scores are percentages.

Models measured
5
Scores recorded
5
Sources
1
IQ weight
0.87

Score distribution

41.3%
Mean
46.3%
Median
17.1%
Std dev
12%
10th percentile
58.6%
90th percentile
12%
Min
12%58.6%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.87 for chat models, measured over 5 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1MiniMax M1 40K· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it.58.6%
2MiniMax M1 80K· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it.56.2%
3GPT 4.1· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim46.3%
4GPT-4.1 mini· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim33.3%
5GPT-4.1 nano· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim12%

Data credits: Data by LLM Stats