Skip to content

Benchmark

Estonian Native Benchmark speaker name extraction F1 (base models)

Multilingual benchmarks compare model performance across languages. Higher scores are better.

Models and systems measured
4
Scores recorded
4
Sources
1
IQ weight
No weight published

Score distribution

0.535
Mean
0.55
Median
0.042
Std dev
0.476
10th percentile
0.591
90th percentile
0.476
Min
0.4760.591
The band covers the middle 80% of scores. The line marks the median.

Results

The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.

Model or systemScore
Llama 3.1 70Bbase modelstable: 8, setting: base models, task: Speakers· Estonian Native Benchmark (arXiv:2510.21193v2) data0.55
Llama 3.1 8Bbase modelstable: 8, setting: base models, task: Speakers· Estonian Native Benchmark (arXiv:2510.21193v2) data0.522
Mistral Nemo Base (12B)base modelstable: 8, setting: base models, task: Speakers· Estonian Native Benchmark (arXiv:2510.21193v2) data0.476
Qwen2 72Bbase modelstable: 8, setting: base models, task: Speakers· Estonian Native Benchmark (arXiv:2510.21193v2) data0.591

Data credits: Lillepalu and Alumäe, Estonian Native Large Language Model Benchmark, arXiv:2510.21193v2. Cited numeric facts only. arXiv non-exclusive distribution is not a results reuse grant. Emitted and committed payloads exclude tasks, examples and generations. Human preference models were available in March 2025; judge evaluation was in March 2025.