Benchmark
Estonian Native Benchmark open-ended Claude 3.7 Sonnet judge win rate
Multilingual benchmarks compare model performance across languages. Higher scores are better.
- Models and systems measured
- 12
- Scores recorded
- 12
- Sources
- 1
- IQ weight
- No weight published
Score distribution
- 0.517
- Mean
- 0.62
- Median
- 0.212
- Std dev
- 0.214
- 10th percentile
- 0.77
- 90th percentile
- 0.205
- Min
Results
The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.
| Model or system | Score |
|---|---|
| Claude 3.7 Sonnetinstruction-tuned modelstable: 12, evaluation: pairwise-open-ended· Estonian Native Benchmark (arXiv:2510.21193v2) data | 0.759 |
| Deepseek V3instruction-tuned modelstable: 12, evaluation: pairwise-open-ended· Estonian Native Benchmark (arXiv:2510.21193v2) data | 0.772 |
| Gemini 2.0 Flash 001instruction-tuned modelstable: 12, evaluation: pairwise-open-ended· Estonian Native Benchmark (arXiv:2510.21193v2) data | 0.77 |
| Gemma 2 27B Itinstruction-tuned modelstable: 12, evaluation: pairwise-open-ended· Estonian Native Benchmark (arXiv:2510.21193v2) data | 0.654 |
| Gemma 2 9B Itinstruction-tuned modelstable: 12, evaluation: pairwise-open-ended· Estonian Native Benchmark (arXiv:2510.21193v2) data | 0.214 |
| GPT-4oinstruction-tuned modelstable: 12, evaluation: pairwise-open-ended· Estonian Native Benchmark (arXiv:2510.21193v2) data | 0.677 |
| GPT-4o Miniinstruction-tuned modelstable: 12, evaluation: pairwise-open-ended· Estonian Native Benchmark (arXiv:2510.21193v2) data | 0.62 |
| Llama 3.1 405B Instructinstruction-tuned modelstable: 12, evaluation: pairwise-open-ended· Estonian Native Benchmark (arXiv:2510.21193v2) data | 0.428 |
| Llama 3.1 70B Instructinstruction-tuned modelstable: 12, evaluation: pairwise-open-ended· Estonian Native Benchmark (arXiv:2510.21193v2) data | 0.473 |
| Llama 3.1 8B Instructinstruction-tuned modelstable: 12, evaluation: pairwise-open-ended· Estonian Native Benchmark (arXiv:2510.21193v2) data | 0.205 |
| Mistral Nemo Instructinstruction-tuned modelstable: 12, evaluation: pairwise-open-ended· Estonian Native Benchmark (arXiv:2510.21193v2) data | 0.226 |
| Qwen2 72B Instructinstruction-tuned modelstable: 12, evaluation: pairwise-open-ended· Estonian Native Benchmark (arXiv:2510.21193v2) data | 0.403 |