Skip to content

Benchmark

CulturALL Easy

Multilingual benchmarks compare model performance across languages. Scores are percentages. Higher is better.

Models measured
8
Scores recorded
15
Sources
1
IQ weight
1.48

Score distribution

78.5%
Mean
87.7%
Median
15.9%
Std dev
52.1%
10th percentile
91.5%
90th percentile
47.9%
Min
47.9%92.6%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.48 for chat models, measured over 8 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Gemini 2.5 Proreasoning: auto, web: yes· CulturALL (arXiv:2604.19262v1, Table 2) data92.6%
2Claude Opus 4reasoning: 1024 tokens, web: no· CulturALL (arXiv:2604.19262v1, Table 2) data91.5%
3GPT 5reasoning: high, web: no· CulturALL (arXiv:2604.19262v1, Table 2) data91.3%
4GPT 5reasoning: medium, web: no· CulturALL (arXiv:2604.19262v1, Table 2) data91.3%
5Claude Opus 4reasoning: disabled, web: no· CulturALL (arXiv:2604.19262v1, Table 2) data90.9%
6Gemini 2.5 Proreasoning: auto, web: no· CulturALL (arXiv:2604.19262v1, Table 2) data90.9%
7GPT 5reasoning: low, web: no· CulturALL (arXiv:2604.19262v1, Table 2) data90.2%
8Gemini 2.5 Proreasoning: 128 tokens, web: yes· CulturALL (arXiv:2604.19262v1, Table 2) data87.7%
9Claude Sonnet 4reasoning: 1024 tokens, web: no· CulturALL (arXiv:2604.19262v1, Table 2) data85.7%
10Gemini 2.5 Flashreasoning: auto, web: yes· CulturALL (arXiv:2604.19262v1, Table 2) data78.3%
11Qwen 3 235B A22Breasoning: hybrid, web: no· CulturALL (arXiv:2604.19262v1, Table 2) data67.2%
12Qwen 3 235B A22Breasoning: hybrid, web: yes· CulturALL (arXiv:2604.19262v1, Table 2) data65.1%
13Qwen Maxreasoning: hybrid, web: yes· CulturALL (arXiv:2604.19262v1, Table 2) data54.7%
14Qwen Maxreasoning: hybrid, web: no· CulturALL (arXiv:2604.19262v1, Table 2) data52.1%
15Qwen3 30B A3Breasoning: hybrid, web: yes· CulturALL (arXiv:2604.19262v1, Table 2) data47.9%

Data credits: Lin et al., CulturALL: Benchmarking Multilingual and Multicultural Competence of LLMs on Grounded Tasks, arXiv:2604.19262v1, CC BY 4.0. Table 2 scores extracted without change. (licence)