Benchmark
RULER · 1M
Long context benchmarks measure how well models use information in long inputs. Scores are percentages. Higher is better.
- Models measured
- 2
- Scores recorded
- 4
- Sources
- 1
- IQ weight
- No weight published
Score distribution
- 92.5%
- Mean
- 91.8%
- Median
- 1.3%
- Std dev
- 91.6%
- 10th percentile
- 94.7%
- 90th percentile
- 91.6%
- Min
Leaderboard
| # | Model | Score |
|---|---|---|
| 1 | Nemotron 3 UltraNeMo Evaluator SDK · 1M· Reported by NVIDIA· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 94.7% |
| 2 | Nemotron 3 Super 120B A12Bofficial blog Figure 3 · rounded· Reported by NVIDIA Developer· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 91.8% |
| 3 | Nemotron 3 Super 120B A12Bofficial HF model card · dedicated RULER container· Reported by NVIDIA· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 91.8% |
| 4 | Nemotron 3 Super 120B A12Btechnical report Table 5/8 · dedicated RULER container· Reported by NVIDIA· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 91.6% |