Skip to content

Benchmark

Estonian Native Benchmark grammar correction Levenshtein similarity (instruction-tuned models)

Multilingual benchmarks compare model performance across languages. Higher scores are better.

Models and systems measured
26
Scores recorded
26
Sources
1
IQ weight
No weight published

Score distribution

0.243
Mean
0.248
Median
0.072
Std dev
0.187
10th percentile
0.321
90th percentile
0.06
Min
0.060.395
The band covers the middle 80% of scores. The line marks the median.

Results

The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.

Model or systemScore
Claude Sonnet 3.7instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Grammar· Estonian Native Benchmark (arXiv:2510.21193v2) data0.391
Claude Sonnet 4instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Grammar· Estonian Native Benchmark (arXiv:2510.21193v2) data0.236
Deepseek V3 (671B-A37B)instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Grammar· Estonian Native Benchmark (arXiv:2510.21193v2) data0.256
EuroLLM-9B-Instructinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Grammar· Estonian Native Benchmark (arXiv:2510.21193v2) data0.248
Gemini 2.0 Flashinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Grammar· Estonian Native Benchmark (arXiv:2510.21193v2) data0.282
Gemini 2.5 Proinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Grammar· Estonian Native Benchmark (arXiv:2510.21193v2) data0.321
Gemma-2-27B-Itinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Grammar· Estonian Native Benchmark (arXiv:2510.21193v2) data0.223
Gemma-2-9B-Itinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Grammar· Estonian Native Benchmark (arXiv:2510.21193v2) data0.182
Gemma-3 27B Instructinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Grammar· Estonian Native Benchmark (arXiv:2510.21193v2) data0.226
Gemma-3-12b-Itinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Grammar· Estonian Native Benchmark (arXiv:2510.21193v2) data0.197
GPT-4oinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Grammar· Estonian Native Benchmark (arXiv:2510.21193v2) data0.395
GPT-4o Miniinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Grammar· Estonian Native Benchmark (arXiv:2510.21193v2) data0.321
GPT-5instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Grammar· Estonian Native Benchmark (arXiv:2510.21193v2) data0.313
GPT-OSS-120Binstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Grammar· Estonian Native Benchmark (arXiv:2510.21193v2) data0.194
Kimi-K2-0905 (1000B-A32B)instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Grammar· Estonian Native Benchmark (arXiv:2510.21193v2) data0.27
Llama-3.1 405B Instructinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Grammar· Estonian Native Benchmark (arXiv:2510.21193v2) data0.286
Llama-3.1 70B Instructinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Grammar· Estonian Native Benchmark (arXiv:2510.21193v2) data0.199
Llama-3.1 8B EstLLM-0825instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Grammar· Estonian Native Benchmark (arXiv:2510.21193v2) data0.275
Llama-3.1 8B Instructinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Grammar· Estonian Native Benchmark (arXiv:2510.21193v2) data0.189
Llama-4 Maverick (400B-A17B)instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Grammar· Estonian Native Benchmark (arXiv:2510.21193v2) data0.261
Llama-4 Scout (109B-A17B)instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Grammar· Estonian Native Benchmark (arXiv:2510.21193v2) data0.27
Mistral Nemo Instruct (12B)instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Grammar· Estonian Native Benchmark (arXiv:2510.21193v2) data0.187
Qwen2-72B-Instructinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Grammar· Estonian Native Benchmark (arXiv:2510.21193v2) data0.06
Qwen3-235B-A22B-2507instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Grammar· Estonian Native Benchmark (arXiv:2510.21193v2) data0.207
Qwen3-4B-Instruct-2507instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Grammar· Estonian Native Benchmark (arXiv:2510.21193v2) data0.136
Qwen3-Next-80B-A3B-Instructinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Grammar· Estonian Native Benchmark (arXiv:2510.21193v2) data0.195

Data credits: Lillepalu and Alumäe, Estonian Native Large Language Model Benchmark, arXiv:2510.21193v2. Cited numeric facts only. arXiv non-exclusive distribution is not a results reuse grant. Emitted and committed payloads exclude tasks, examples and generations. Human preference models were available in March 2025; judge evaluation was in March 2025.