Skip to content

Benchmark

Estonian Native Benchmark word meaning accuracy (base models)

Multilingual benchmarks compare model performance across languages. Higher scores are better.

Models and systems measured
6
Scores recorded
6
Sources
1
IQ weight
No weight published

Score distribution

0.267
Mean
0.259
Median
0.076
Std dev
0.188
10th percentile
0.379
90th percentile
0.182
Min
0.1820.379
The band covers the middle 80% of scores. The line marks the median.

Results

The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.

Model or systemScore
Gemma 2 27Bbase modelstable: 8, setting: base models, task: Words· Estonian Native Benchmark (arXiv:2510.21193v2) data0.354
Gemma 2 9Bbase modelstable: 8, setting: base models, task: Words· Estonian Native Benchmark (arXiv:2510.21193v2) data0.259
Llama 3.1 70Bbase modelstable: 8, setting: base models, task: Words· Estonian Native Benchmark (arXiv:2510.21193v2) data0.379
Llama 3.1 8Bbase modelstable: 8, setting: base models, task: Words· Estonian Native Benchmark (arXiv:2510.21193v2) data0.182
Mistral Nemo Base (12B)base modelstable: 8, setting: base models, task: Words· Estonian Native Benchmark (arXiv:2510.21193v2) data0.237
Qwen2 72Bbase modelstable: 8, setting: base models, task: Words· Estonian Native Benchmark (arXiv:2510.21193v2) data0.188

Data credits: Lillepalu and Alumäe, Estonian Native Large Language Model Benchmark, arXiv:2510.21193v2. Cited numeric facts only. arXiv non-exclusive distribution is not a results reuse grant. Emitted and committed payloads exclude tasks, examples and generations. Human preference models were available in March 2025; judge evaluation was in March 2025.