Benchmark
Estonian Native Benchmark open-ended human preference win rate
Multilingual benchmarks compare model performance across languages. Higher scores are better.
- Models and systems measured
- 12
- Scores recorded
- 12
- Sources
- 1
- IQ weight
- No weight published
Score distribution
- 0.528
- Mean
- 0.545
- Median
- 0.161
- Std dev
- 0.327
- 10th percentile
- 0.746
- 90th percentile
- 0.211
- Min
Results
The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.
| Model or system | Score |
|---|---|
| Claude 3.7 Sonnetinstruction-tuned modelstable: 10, evaluation: pairwise-open-ended· Estonian Native Benchmark (arXiv:2510.21193v2) data | 0.754 |
| Deepseek Chat V3instruction-tuned modelstable: 10, evaluation: pairwise-open-ended· Estonian Native Benchmark (arXiv:2510.21193v2) data | 0.746 |
| Gemini 2.0 Flash 001instruction-tuned modelstable: 10, evaluation: pairwise-open-ended· Estonian Native Benchmark (arXiv:2510.21193v2) data | 0.699 |
| Gemma 2 27B Itinstruction-tuned modelstable: 10, evaluation: pairwise-open-ended· Estonian Native Benchmark (arXiv:2510.21193v2) data | 0.642 |
| Gemma 2 9B Itinstruction-tuned modelstable: 10, evaluation: pairwise-open-ended· Estonian Native Benchmark (arXiv:2510.21193v2) data | 0.211 |
| GPT-4oinstruction-tuned modelstable: 10, evaluation: pairwise-open-ended· Estonian Native Benchmark (arXiv:2510.21193v2) data | 0.545 |
| GPT-4o Miniinstruction-tuned modelstable: 10, evaluation: pairwise-open-ended· Estonian Native Benchmark (arXiv:2510.21193v2) data | 0.533 |
| Llama 3.1 405B Instructinstruction-tuned modelstable: 10, evaluation: pairwise-open-ended· Estonian Native Benchmark (arXiv:2510.21193v2) data | 0.5 |
| Llama 3.1 70B Instructinstruction-tuned modelstable: 10, evaluation: pairwise-open-ended· Estonian Native Benchmark (arXiv:2510.21193v2) data | 0.552 |
| Llama 3.1 8B Instructinstruction-tuned modelstable: 10, evaluation: pairwise-open-ended· Estonian Native Benchmark (arXiv:2510.21193v2) data | 0.389 |
| Mistral Nemo Instructinstruction-tuned modelstable: 10, evaluation: pairwise-open-ended· Estonian Native Benchmark (arXiv:2510.21193v2) data | 0.327 |
| Qwen2 72B Instructinstruction-tuned modelstable: 10, evaluation: pairwise-open-ended· Estonian Native Benchmark (arXiv:2510.21193v2) data | 0.442 |