Skip to content

Benchmark

Estonian Native Benchmark mean score (instruction-tuned models)

Multilingual benchmarks compare model performance across languages. Higher scores are better.

Models and systems measured
26
Scores recorded
26
Sources
1
IQ weight
No weight published

Score distribution

0.435
Mean
0.438
Median
0.119
Std dev
0.28
10th percentile
0.616
90th percentile
0.212
Min
0.2120.638
The band covers the middle 80% of scores. The line marks the median.

Results

The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.

Model or systemScore
Claude Sonnet 3.7instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Mean· Estonian Native Benchmark (arXiv:2510.21193v2) data0.626
Claude Sonnet 4instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Mean· Estonian Native Benchmark (arXiv:2510.21193v2) data0.526
Deepseek V3 (671B-A37B)instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Mean· Estonian Native Benchmark (arXiv:2510.21193v2) data0.482
EuroLLM-9B-Instructinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Mean· Estonian Native Benchmark (arXiv:2510.21193v2) data0.409
Gemini 2.0 Flashinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Mean· Estonian Native Benchmark (arXiv:2510.21193v2) data0.569
Gemini 2.5 Proinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Mean· Estonian Native Benchmark (arXiv:2510.21193v2) data0.638
Gemma-2-27B-Itinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Mean· Estonian Native Benchmark (arXiv:2510.21193v2) data0.365
Gemma-2-9B-Itinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Mean· Estonian Native Benchmark (arXiv:2510.21193v2) data0.29
Gemma-3 27B Instructinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Mean· Estonian Native Benchmark (arXiv:2510.21193v2) data0.386
Gemma-3-12b-Itinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Mean· Estonian Native Benchmark (arXiv:2510.21193v2) data0.354
GPT-4oinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Mean· Estonian Native Benchmark (arXiv:2510.21193v2) data0.616
GPT-4o Miniinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Mean· Estonian Native Benchmark (arXiv:2510.21193v2) data0.438
GPT-5instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Mean· Estonian Native Benchmark (arXiv:2510.21193v2) data0.587
GPT-OSS-120Binstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Mean· Estonian Native Benchmark (arXiv:2510.21193v2) data0.372
Kimi-K2-0905 (1000B-A32B)instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Mean· Estonian Native Benchmark (arXiv:2510.21193v2) data0.533
Llama-3.1 405B Instructinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Mean· Estonian Native Benchmark (arXiv:2510.21193v2) data0.475
Llama-3.1 70B Instructinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Mean· Estonian Native Benchmark (arXiv:2510.21193v2) data0.401
Llama-3.1 8B EstLLM-0825instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Mean· Estonian Native Benchmark (arXiv:2510.21193v2) data0.454
Llama-3.1 8B Instructinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Mean· Estonian Native Benchmark (arXiv:2510.21193v2) data0.244
Llama-4 Maverick (400B-A17B)instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Mean· Estonian Native Benchmark (arXiv:2510.21193v2) data0.51
Llama-4 Scout (109B-A17B)instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Mean· Estonian Native Benchmark (arXiv:2510.21193v2) data0.488
Mistral Nemo Instruct (12B)instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Mean· Estonian Native Benchmark (arXiv:2510.21193v2) data0.271
Qwen2-72B-Instructinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Mean· Estonian Native Benchmark (arXiv:2510.21193v2) data0.28
Qwen3-235B-A22B-2507instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Mean· Estonian Native Benchmark (arXiv:2510.21193v2) data0.422
Qwen3-4B-Instruct-2507instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Mean· Estonian Native Benchmark (arXiv:2510.21193v2) data0.212
Qwen3-Next-80B-A3B-Instructinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Mean· Estonian Native Benchmark (arXiv:2510.21193v2) data0.356

Data credits: Lillepalu and Alumäe, Estonian Native Large Language Model Benchmark, arXiv:2510.21193v2. Cited numeric facts only. arXiv non-exclusive distribution is not a results reuse grant. Emitted and committed payloads exclude tasks, examples and generations. Human preference models were available in March 2025; judge evaluation was in March 2025.