Skip to content

Benchmark

CulturALL All

Multilingual benchmarks compare model performance across languages. Scores are percentages. Higher is better.

Models measured
8
Scores recorded
15
Sources
1
IQ weight
1.09

Score distribution

31.7%
Mean
36.5%
Median
8.4%
Std dev
19%
10th percentile
39.3%
90th percentile
17.6%
Min
17.6%44.5%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.09 for chat models, measured over 8 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Gemini 2.5 Proreasoning: auto, web: yes· CulturALL (arXiv:2604.19262v1, Table 2) data44.5%
2Gemini 2.5 Proreasoning: 128 tokens, web: yes· CulturALL (arXiv:2604.19262v1, Table 2) data39.3%
3Gemini 2.5 Proreasoning: auto, web: no· CulturALL (arXiv:2604.19262v1, Table 2) data37.9%
4GPT 5reasoning: high, web: no· CulturALL (arXiv:2604.19262v1, Table 2) data37.6%
5GPT 5reasoning: low, web: no· CulturALL (arXiv:2604.19262v1, Table 2) data37.2%
6GPT 5reasoning: medium, web: no· CulturALL (arXiv:2604.19262v1, Table 2) data37.2%
7Claude Opus 4reasoning: 1024 tokens, web: no· CulturALL (arXiv:2604.19262v1, Table 2) data36.7%
8Claude Opus 4reasoning: disabled, web: no· CulturALL (arXiv:2604.19262v1, Table 2) data36.5%
9Gemini 2.5 Flashreasoning: auto, web: yes· CulturALL (arXiv:2604.19262v1, Table 2) data33.7%
10Claude Sonnet 4reasoning: 1024 tokens, web: no· CulturALL (arXiv:2604.19262v1, Table 2) data32.8%
11Qwen 3 235B A22Breasoning: hybrid, web: no· CulturALL (arXiv:2604.19262v1, Table 2) data23.7%
12Qwen 3 235B A22Breasoning: hybrid, web: yes· CulturALL (arXiv:2604.19262v1, Table 2) data22.5%
13Qwen Maxreasoning: hybrid, web: yes· CulturALL (arXiv:2604.19262v1, Table 2) data19.3%
14Qwen Maxreasoning: hybrid, web: no· CulturALL (arXiv:2604.19262v1, Table 2) data19%
15Qwen3 30B A3Breasoning: hybrid, web: yes· CulturALL (arXiv:2604.19262v1, Table 2) data17.6%

Data credits: Lin et al., CulturALL: Benchmarking Multilingual and Multicultural Competence of LLMs on Grounded Tasks, arXiv:2604.19262v1, CC BY 4.0. Table 2 scores extracted without change. (licence)