Benchmarks
Long context benchmarks
These benchmarks measure how well models use information in long inputs.
We track 130 benchmarks in this topic. Main benchmarks are listed by how many models they cover, most first. Where a benchmark has separate results for a subset of its tasks, those results appear right under it. Click one for its leaderboard.
Showing 1–100 of 130 benchmarks.
- FictionLiveBench 054 models
- FictionLiveBench 16k54 models
- FictionLiveBench 1k54 models
- FictionLiveBench 2k54 models
- FictionLiveBench 40054 models
- FictionLiveBench 4k54 models
- FictionLiveBench 8k54 models
- FictionLiveBench 32k47 models
- FictionLiveBench 60k47 models
- FictionLiveBench 120k39 models
- RULER 64K29 models
- RULER 4K27 models
- MRCR v2 (8-needle)26 models
- RULER 16K26 models
- RULER 8K26 models
- RULER Avg26 models
- RULER wAvg (dec)20 models
- RULER wAvg (inc)20 models
- FictionLiveBench 192k18 models
- BABILong avg(qa1-5) 0k16 models
- BABILong avg(qa1-5) 1k16 models
- BABILong avg(qa1-5) 2k16 models
- BABILong avg(qa1-5) 4k16 models
- BABILong avg(qa1-5) 8k16 models
- BABILong avg(qa1-5) 16k15 models
- BABILong avg(qa1-5) 32k15 models
- BABILong qa1 0k14 models
- BABILong qa1 1k14 models
- BABILong qa1 2k14 models
- BABILong qa1 4k14 models
- BABILong qa1 8k14 models
- BABILong qa2 0k14 models
- BABILong qa2 1k14 models
- BABILong qa2 2k14 models
- BABILong qa2 4k14 models
- BABILong qa2 8k14 models
- BABILong qa3 0k14 models
- BABILong qa3 1k14 models
- BABILong qa3 2k14 models
- BABILong qa3 4k14 models
- BABILong qa3 8k14 models
- BABILong qa4 0k14 models
- BABILong qa4 1k14 models
- BABILong qa4 2k14 models
- BABILong qa4 4k14 models
- BABILong qa4 8k14 models
- BABILong qa5 0k14 models
- BABILong qa5 1k14 models
- BABILong qa5 2k14 models
- BABILong qa5 4k14 models
- BABILong qa5 8k14 models
- BABILong qa1 16k13 models
- BABILong qa1 32k13 models
- BABILong qa2 16k13 models
- BABILong qa2 32k13 models
- BABILong qa3 16k13 models
- BABILong qa3 32k13 models
- BABILong qa4 16k13 models
- BABILong qa4 32k13 models
- BABILong qa5 16k13 models
- BABILong qa5 32k13 models
- BABILong avg(qa1-5) 64k12 models
- BABILong avg(qa1-5) 128k10 models
- BABILong qa1 64k10 models
- BABILong qa2 64k10 models
- BABILong qa3 64k10 models
- BABILong qa4 64k10 models
- BABILong qa5 64k10 models
- OpenAI-MRCR: 2 needle 128k9 models
- BABILong qa1 128k8 models
- BABILong qa2 128k8 models
- BABILong qa3 128k8 models
- BABILong qa4 128k8 models
- BABILong qa5 128k8 models
- OpenAI-MRCR: 2 needle 1M5 models
- BABILong qa1 512k4 models
- MRCR v2 (8-needle, 512K-1M)4 models
- RULER4 models
- BABILong avg(qa1-5) 10M3 models
- BABILong avg(qa1-5) 1M3 models
- BABILong avg(qa1-5) 512k3 models