Benchmark
BABILong avg(qa1-5) 10M
Long context benchmarks measure how well models use information in long inputs. Scores are percentages. Higher is better.
All long context benchmarksScores from BABILong v0 data docs.
- Models and systems measured
- 3
- Scores recorded
- 3
- Sources
- 1
- IQ weight
- No weight published
Score distribution
- 49.1%
- Mean
- 37%
- Median
- 19.5%
- Std dev
- 33.8%
- 10th percentile
- 76.6%
- 90th percentile
- 33.8%
- Min
Results
The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.
| Model or system | Score |
|---|---|
| ~ ARMT (137M) fine-tuneavg, 10M· BABILong v0 data docs | 76.6% |
| ~ RMT (137M) fine-tuneavg, 10M· BABILong v0 data docs | 33.8% |
| Llama3-ChatQA-1.5-8B + RAGavg, 10M· BABILong v0 data docs | 37% |
Data credits: RMT-team, Kuratov et al. Official v0 numeric CSV; results licence not separately stated.