Skip to content

Benchmark

Estonian Native Benchmark grammar correction Levenshtein similarity (base models)

Multilingual benchmarks compare model performance across languages. Higher scores are better.

Models and systems measured
6
Scores recorded
6
Sources
1
IQ weight
No weight published

Score distribution

0.267
Mean
0.263
Median
0.024
Std dev
0.245
10th percentile
0.306
90th percentile
0.242
Min
0.2420.306
The band covers the middle 80% of scores. The line marks the median.

Results

The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.

Model or systemScore
Gemma 2 27Bbase modelstable: 8, setting: base models, task: Grammar· Estonian Native Benchmark (arXiv:2510.21193v2) data0.306
Gemma 2 9Bbase modelstable: 8, setting: base models, task: Grammar· Estonian Native Benchmark (arXiv:2510.21193v2) data0.263
Llama 3.1 70Bbase modelstable: 8, setting: base models, task: Grammar· Estonian Native Benchmark (arXiv:2510.21193v2) data0.291
Llama 3.1 8Bbase modelstable: 8, setting: base models, task: Grammar· Estonian Native Benchmark (arXiv:2510.21193v2) data0.242
Mistral Nemo Base (12B)base modelstable: 8, setting: base models, task: Grammar· Estonian Native Benchmark (arXiv:2510.21193v2) data0.245
Qwen2 72Bbase modelstable: 8, setting: base models, task: Grammar· Estonian Native Benchmark (arXiv:2510.21193v2) data0.257

Data credits: Lillepalu and Alumäe, Estonian Native Large Language Model Benchmark, arXiv:2510.21193v2. Cited numeric facts only. arXiv non-exclusive distribution is not a results reuse grant. Emitted and committed payloads exclude tasks, examples and generations. Human preference models were available in March 2025; judge evaluation was in March 2025.