Skip to content

Benchmark

Estonian Native Benchmark trivia accuracy (instruction-tuned models)

Multilingual benchmarks compare model performance across languages. Higher scores are better.

Models and systems measured
26
Scores recorded
26
Sources
1
IQ weight
No weight published

Score distribution

0.547
Mean
0.538
Median
0.139
Std dev
0.386
10th percentile
0.77
90th percentile
0.329
Min
0.3290.889
The band covers the middle 80% of scores. The line marks the median.

Results

The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.

Model or systemScore
Claude Sonnet 3.7instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Trivia· Estonian Native Benchmark (arXiv:2510.21193v2) data0.816
Claude Sonnet 4instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Trivia· Estonian Native Benchmark (arXiv:2510.21193v2) data0.638
Deepseek V3 (671B-A37B)instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Trivia· Estonian Native Benchmark (arXiv:2510.21193v2) data0.619
EuroLLM-9B-Instructinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Trivia· Estonian Native Benchmark (arXiv:2510.21193v2) data0.508
Gemini 2.0 Flashinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Trivia· Estonian Native Benchmark (arXiv:2510.21193v2) data0.635
Gemini 2.5 Proinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Trivia· Estonian Native Benchmark (arXiv:2510.21193v2) data0.889
Gemma-2-27B-Itinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Trivia· Estonian Native Benchmark (arXiv:2510.21193v2) data0.408
Gemma-2-9B-Itinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Trivia· Estonian Native Benchmark (arXiv:2510.21193v2) data0.386
Gemma-3 27B Instructinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Trivia· Estonian Native Benchmark (arXiv:2510.21193v2) data0.459
Gemma-3-12b-Itinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Trivia· Estonian Native Benchmark (arXiv:2510.21193v2) data0.426
GPT-4oinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Trivia· Estonian Native Benchmark (arXiv:2510.21193v2) data0.77
GPT-4o Miniinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Trivia· Estonian Native Benchmark (arXiv:2510.21193v2) data0.525
GPT-5instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Trivia· Estonian Native Benchmark (arXiv:2510.21193v2) data0.755
GPT-OSS-120Binstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Trivia· Estonian Native Benchmark (arXiv:2510.21193v2) data0.54
Kimi-K2-0905 (1000B-A32B)instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Trivia· Estonian Native Benchmark (arXiv:2510.21193v2) data0.598
Llama-3.1 405B Instructinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Trivia· Estonian Native Benchmark (arXiv:2510.21193v2) data0.53
Llama-3.1 70B Instructinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Trivia· Estonian Native Benchmark (arXiv:2510.21193v2) data0.549
Llama-3.1 8B EstLLM-0825instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Trivia· Estonian Native Benchmark (arXiv:2510.21193v2) data0.586
Llama-3.1 8B Instructinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Trivia· Estonian Native Benchmark (arXiv:2510.21193v2) data0.385
Llama-4 Maverick (400B-A17B)instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Trivia· Estonian Native Benchmark (arXiv:2510.21193v2) data0.549
Llama-4 Scout (109B-A17B)instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Trivia· Estonian Native Benchmark (arXiv:2510.21193v2) data0.538
Mistral Nemo Instruct (12B)instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Trivia· Estonian Native Benchmark (arXiv:2510.21193v2) data0.386
Qwen2-72B-Instructinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Trivia· Estonian Native Benchmark (arXiv:2510.21193v2) data0.443
Qwen3-235B-A22B-2507instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Trivia· Estonian Native Benchmark (arXiv:2510.21193v2) data0.51
Qwen3-4B-Instruct-2507instruction-tuned modelstable: 9, setting: instruction-tuned models, task: Trivia· Estonian Native Benchmark (arXiv:2510.21193v2) data0.329
Qwen3-Next-80B-A3B-Instructinstruction-tuned modelstable: 9, setting: instruction-tuned models, task: Trivia· Estonian Native Benchmark (arXiv:2510.21193v2) data0.449

Data credits: Lillepalu and Alumäe, Estonian Native Large Language Model Benchmark, arXiv:2510.21193v2. Cited numeric facts only. arXiv non-exclusive distribution is not a results reuse grant. Emitted and committed payloads exclude tasks, examples and generations. Human preference models were available in March 2025; judge evaluation was in March 2025.