Skip to content

Benchmark

Estonian Native Benchmark declension accuracy (instruction-tuned models)

Multilingual benchmarks compare model performance across languages. Higher scores are better.

Models and systems measured
26
Scores recorded
26
Sources
1
IQ weight
No weight published

Score distribution

0.582
Mean
0.631
Median
0.279
Std dev
0.274
10th percentile
0.937
90th percentile
0.056
Min
0.0560.955
The band covers the middle 80% of scores. The line marks the median.

Results

The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.

Model or systemScore
Claude Sonnet 3.7instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Declension· Estonian Native Benchmark (arXiv:2510.21193v2) data0.937
Claude Sonnet 4instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Declension· Estonian Native Benchmark (arXiv:2510.21193v2) data0.822
Deepseek V3 (671B-A37B)instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Declension· Estonian Native Benchmark (arXiv:2510.21193v2) data0.631
EuroLLM-9B-Instructinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Declension· Estonian Native Benchmark (arXiv:2510.21193v2) data0.644
Gemini 2.0 Flashinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Declension· Estonian Native Benchmark (arXiv:2510.21193v2) data0.932
Gemini 2.5 Proinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Declension· Estonian Native Benchmark (arXiv:2510.21193v2) data0.955
Gemma-2-27B-Itinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Declension· Estonian Native Benchmark (arXiv:2510.21193v2) data0.406
Gemma-2-9B-Itinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Declension· Estonian Native Benchmark (arXiv:2510.21193v2) data0.308
Gemma-3 27B Instructinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Declension· Estonian Native Benchmark (arXiv:2510.21193v2) data0.404
Gemma-3-12b-Itinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Declension· Estonian Native Benchmark (arXiv:2510.21193v2) data0.356
GPT-4oinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Declension· Estonian Native Benchmark (arXiv:2510.21193v2) data0.948
GPT-4o Miniinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Declension· Estonian Native Benchmark (arXiv:2510.21193v2) data0.554
GPT-5instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Declension· Estonian Native Benchmark (arXiv:2510.21193v2) data0.934
GPT-OSS-120Binstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Declension· Estonian Native Benchmark (arXiv:2510.21193v2) data0.356
Kimi-K2-0905 (1000B-A32B)instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Declension· Estonian Native Benchmark (arXiv:2510.21193v2) data0.876
Llama-3.1 405B Instructinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Declension· Estonian Native Benchmark (arXiv:2510.21193v2) data0.68
Llama-3.1 70B Instructinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Declension· Estonian Native Benchmark (arXiv:2510.21193v2) data0.464
Llama-3.1 8B EstLLM-0825instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Declension· Estonian Native Benchmark (arXiv:2510.21193v2) data0.811
Llama-3.1 8B Instructinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Declension· Estonian Native Benchmark (arXiv:2510.21193v2) data0.099
Llama-4 Maverick (400B-A17B)instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Declension· Estonian Native Benchmark (arXiv:2510.21193v2) data0.828
Llama-4 Scout (109B-A17B)instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Declension· Estonian Native Benchmark (arXiv:2510.21193v2) data0.766
Mistral Nemo Instruct (12B)instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Declension· Estonian Native Benchmark (arXiv:2510.21193v2) data0.274
Qwen2-72B-Instructinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Declension· Estonian Native Benchmark (arXiv:2510.21193v2) data0.177
Qwen3-235B-A22B-2507instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Declension· Estonian Native Benchmark (arXiv:2510.21193v2) data0.555
Qwen3-4B-Instruct-2507instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Declension· Estonian Native Benchmark (arXiv:2510.21193v2) data0.056
Qwen3-Next-80B-A3B-Instructinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Declension· Estonian Native Benchmark (arXiv:2510.21193v2) data0.362

Data credits: Lillepalu and Alumäe, Estonian Native Large Language Model Benchmark, arXiv:2510.21193v2. Cited numeric facts only. arXiv non-exclusive distribution is not a results reuse grant. Emitted and committed payloads exclude tasks, examples and generations. Human preference models were available in March 2025; judge evaluation was in March 2025.