Skip to content

Benchmark

Estonian Native Benchmark national exams accuracy (base models)

Multilingual benchmarks compare model performance across languages. Higher scores are better.

Models and systems measured
6
Scores recorded
6
Sources
1
IQ weight
No weight published

Score distribution

0.588
Mean
0.659
Median
0.105
Std dev
0.487
10th percentile
0.693
90th percentile
0.405
Min
0.4050.693
The band covers the middle 80% of scores. The line marks the median.

Results

The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.

Model or systemScore
Gemma 2 27Bbase modelstable: 8, setting: base models, task: Exams· Estonian Native Benchmark (arXiv:2510.21193v2) data0.693
Gemma 2 9Bbase modelstable: 8, setting: base models, task: Exams· Estonian Native Benchmark (arXiv:2510.21193v2) data0.616
Llama 3.1 70Bbase modelstable: 8, setting: base models, task: Exams· Estonian Native Benchmark (arXiv:2510.21193v2) data0.667
Llama 3.1 8Bbase modelstable: 8, setting: base models, task: Exams· Estonian Native Benchmark (arXiv:2510.21193v2) data0.405
Mistral Nemo Base (12B)base modelstable: 8, setting: base models, task: Exams· Estonian Native Benchmark (arXiv:2510.21193v2) data0.487
Qwen2 72Bbase modelstable: 8, setting: base models, task: Exams· Estonian Native Benchmark (arXiv:2510.21193v2) data0.659

Data credits: Lillepalu and Alumäe, Estonian Native Large Language Model Benchmark, arXiv:2510.21193v2. Cited numeric facts only. arXiv non-exclusive distribution is not a results reuse grant. Emitted and committed payloads exclude tasks, examples and generations. Human preference models were available in March 2025; judge evaluation was in March 2025.