Skip to content

Benchmark

Estonian Native Benchmark news summarization ROUGE-L (base models)

Multilingual benchmarks compare model performance across languages. Higher scores are better.

Models and systems measured
6
Scores recorded
6
Sources
1
IQ weight
No weight published

Score distribution

0.145
Mean
0.157
Median
0.015
Std dev
0.13
10th percentile
0.164
90th percentile
0.126
Min
0.1260.164
The band covers the middle 80% of scores. The line marks the median.

Results

The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.

Model or systemScore
Gemma 2 27Bbase modelstable: 8, setting: base models, task: News· Estonian Native Benchmark (arXiv:2510.21193v2) data0.164
Gemma 2 9Bbase modelstable: 8, setting: base models, task: News· Estonian Native Benchmark (arXiv:2510.21193v2) data0.157
Llama 3.1 70Bbase modelstable: 8, setting: base models, task: News· Estonian Native Benchmark (arXiv:2510.21193v2) data0.16
Llama 3.1 8Bbase modelstable: 8, setting: base models, task: News· Estonian Native Benchmark (arXiv:2510.21193v2) data0.126
Mistral Nemo Base (12B)base modelstable: 8, setting: base models, task: News· Estonian Native Benchmark (arXiv:2510.21193v2) data0.13
Qwen2 72Bbase modelstable: 8, setting: base models, task: News· Estonian Native Benchmark (arXiv:2510.21193v2) data0.134

Data credits: Lillepalu and Alumäe, Estonian Native Large Language Model Benchmark, arXiv:2510.21193v2. Cited numeric facts only. arXiv non-exclusive distribution is not a results reuse grant. Emitted and committed payloads exclude tasks, examples and generations. Human preference models were available in March 2025; judge evaluation was in March 2025.