Skip to content

Benchmark

CulturALL Medium

Multilingual benchmarks compare model performance across languages. Scores are percentages. Higher is better.

Models measured
8
Scores recorded
15
Sources
1
IQ weight
1.02

Score distribution

46.6%
Mean
55.7%
Median
14.4%
Std dev
24%
10th percentile
59.6%
90th percentile
23.9%
Min
23.9%65.7%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.02 for chat models, measured over 8 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Gemini 2.5 Proreasoning: auto, web: yes· CulturALL (arXiv:2604.19262v1, Table 2) data65.7%
2Gemini 2.5 Proreasoning: auto, web: no· CulturALL (arXiv:2604.19262v1, Table 2) data59.6%
3GPT 5reasoning: low, web: no· CulturALL (arXiv:2604.19262v1, Table 2) data59%
4GPT 5reasoning: medium, web: no· CulturALL (arXiv:2604.19262v1, Table 2) data58.3%
5GPT 5reasoning: high, web: no· CulturALL (arXiv:2604.19262v1, Table 2) data57.7%
6Claude Opus 4reasoning: disabled, web: no· CulturALL (arXiv:2604.19262v1, Table 2) data56.4%
7Claude Opus 4reasoning: 1024 tokens, web: no· CulturALL (arXiv:2604.19262v1, Table 2) data56%
8Gemini 2.5 Proreasoning: 128 tokens, web: yes· CulturALL (arXiv:2604.19262v1, Table 2) data55.7%
9Gemini 2.5 Flashreasoning: auto, web: yes· CulturALL (arXiv:2604.19262v1, Table 2) data46.7%
10Claude Sonnet 4reasoning: 1024 tokens, web: no· CulturALL (arXiv:2604.19262v1, Table 2) data46.4%
11Qwen 3 235B A22Breasoning: hybrid, web: no· CulturALL (arXiv:2604.19262v1, Table 2) data33.6%
12Qwen 3 235B A22Breasoning: hybrid, web: yes· CulturALL (arXiv:2604.19262v1, Table 2) data31.4%
13Qwen Maxreasoning: hybrid, web: yes· CulturALL (arXiv:2604.19262v1, Table 2) data24.7%
14Qwen Maxreasoning: hybrid, web: no· CulturALL (arXiv:2604.19262v1, Table 2) data24%
15Qwen3 30B A3Breasoning: hybrid, web: yes· CulturALL (arXiv:2604.19262v1, Table 2) data23.9%

Data credits: Lin et al., CulturALL: Benchmarking Multilingual and Multicultural Competence of LLMs on Grounded Tasks, arXiv:2604.19262v1, CC BY 4.0. Table 2 scores extracted without change. (licence)