Skip to content

Benchmark

Estonian Native Benchmark national exams accuracy (instruction-tuned models)

Multilingual benchmarks compare model performance across languages. Higher scores are better.

Models and systems measured
26
Scores recorded
26
Sources
1
IQ weight
No weight published

Score distribution

0.764
Mean
0.784
Median
0.119
Std dev
0.57
10th percentile
0.9
90th percentile
0.542
Min
0.5420.905
The band covers the middle 80% of scores. The line marks the median.

Results

The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.

Model or systemScore
Claude Sonnet 3.7instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Exams· Estonian Native Benchmark (arXiv:2510.21193v2) data0.905
Claude Sonnet 4instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Exams· Estonian Native Benchmark (arXiv:2510.21193v2) data0.876
Deepseek V3 (671B-A37B)instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Exams· Estonian Native Benchmark (arXiv:2510.21193v2) data0.82
EuroLLM-9B-Instructinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Exams· Estonian Native Benchmark (arXiv:2510.21193v2) data0.61
Gemini 2.0 Flashinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Exams· Estonian Native Benchmark (arXiv:2510.21193v2) data0.9
Gemini 2.5 Proinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Exams· Estonian Native Benchmark (arXiv:2510.21193v2) data0.897
Gemma-2-27B-Itinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Exams· Estonian Native Benchmark (arXiv:2510.21193v2) data0.76
Gemma-2-9B-Itinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Exams· Estonian Native Benchmark (arXiv:2510.21193v2) data0.556
Gemma-3 27B Instructinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Exams· Estonian Native Benchmark (arXiv:2510.21193v2) data0.756
Gemma-3-12b-Itinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Exams· Estonian Native Benchmark (arXiv:2510.21193v2) data0.736
GPT-4oinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Exams· Estonian Native Benchmark (arXiv:2510.21193v2) data0.9
GPT-4o Miniinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Exams· Estonian Native Benchmark (arXiv:2510.21193v2) data0.784
GPT-5instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Exams· Estonian Native Benchmark (arXiv:2510.21193v2) data0.875
GPT-OSS-120Binstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Exams· Estonian Native Benchmark (arXiv:2510.21193v2) data0.777
Kimi-K2-0905 (1000B-A32B)instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Exams· Estonian Native Benchmark (arXiv:2510.21193v2) data0.88
Llama-3.1 405B Instructinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Exams· Estonian Native Benchmark (arXiv:2510.21193v2) data0.817
Llama-3.1 70B Instructinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Exams· Estonian Native Benchmark (arXiv:2510.21193v2) data0.76
Llama-3.1 8B EstLLM-0825instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Exams· Estonian Native Benchmark (arXiv:2510.21193v2) data0.575
Llama-3.1 8B Instructinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Exams· Estonian Native Benchmark (arXiv:2510.21193v2) data0.542
Llama-4 Maverick (400B-A17B)instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Exams· Estonian Native Benchmark (arXiv:2510.21193v2) data0.841
Llama-4 Scout (109B-A17B)instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Exams· Estonian Native Benchmark (arXiv:2510.21193v2) data0.824
Mistral Nemo Instruct (12B)instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Exams· Estonian Native Benchmark (arXiv:2510.21193v2) data0.57
Qwen2-72B-Instructinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Exams· Estonian Native Benchmark (arXiv:2510.21193v2) data0.739
Qwen3-235B-A22B-2507instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Exams· Estonian Native Benchmark (arXiv:2510.21193v2) data0.829
Qwen3-4B-Instruct-2507instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Exams· Estonian Native Benchmark (arXiv:2510.21193v2) data0.553
Qwen3-Next-80B-A3B-Instructinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Exams· Estonian Native Benchmark (arXiv:2510.21193v2) data0.776

Data credits: Lillepalu and Alumäe, Estonian Native Large Language Model Benchmark, arXiv:2510.21193v2. Cited numeric facts only. arXiv non-exclusive distribution is not a results reuse grant. Emitted and committed payloads exclude tasks, examples and generations. Human preference models were available in March 2025; judge evaluation was in March 2025.