Skip to content

Benchmark

LaoBench Lao-500 win rate vs GPT-5-High (Qwen3-Max judge)

Multilingual benchmarks compare model performance across languages. Scores are percentages. Higher is better.

Models and systems measured
11
Scores recorded
12
Sources
1
IQ weight
0.33

Score distribution

48.4%
Mean
48.9%
Median
2.4%
Std dev
45.5%
10th percentile
51.8%
90th percentile
45.2%
Min
45.2%52.8%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.33 for chat models, measured over 7 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#Model or systemScore
1Qwen3 235B A22B Instruct 2507table: 10, judge: Qwen3-Max, mode: single· LaoBench (arXiv:2511.11334v3) data51.8%
2Claude Sonnet 4.5table: 10, judge: Qwen3-Max, mode: single· LaoBench (arXiv:2511.11334v3) data50.1%
3Gemini 2.5 Protable: 10, judge: Qwen3-Max, mode: single· LaoBench (arXiv:2511.11334v3) data48.9%
4Claude Opus 4.1table: 10, judge: Qwen3-Max, mode: single· LaoBench (arXiv:2511.11334v3) data47.6%
5DeepSeek V3.2 Exptable: 10, judge: Qwen3-Max, mode: Thinking· LaoBench (arXiv:2511.11334v3) data47.3%
6DeepSeek V3.2 Exptable: 10, judge: Qwen3-Max, mode: Non-Thinking· LaoBench (arXiv:2511.11334v3) data46.4%
7Qwen3 Next 80B A3B Instructtable: 10, judge: Qwen3-Max, mode: single· LaoBench (arXiv:2511.11334v3) data45.7%
8Qwen 3 235B A22Btable: 10, judge: Qwen3-Max, mode: single· LaoBench (arXiv:2511.11334v3) data45.5%

Not ranked

The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.

Model or systemScore
GPT 5table: 10, judge: Qwen3-Max, mode: single-self-baseline· LaoBench (arXiv:2511.11334v3) data49.9%
Qwen3 Maxtable: 10, judge: Qwen3-Max, mode: single· LaoBench (arXiv:2511.11334v3) data52.8%
Qwen3-PlusNon-Thinkingtable: 10, judge: Qwen3-Max, mode: Non-Thinking· LaoBench (arXiv:2511.11334v3) data49.4%
Qwen3-PlusThinkingtable: 10, judge: Qwen3-Max, mode: Thinking· LaoBench (arXiv:2511.11334v3) data45.2%

Data credits: LaoBench, BAAI et al., arXiv:2511.11334v3. Cited numeric results only. arXiv non-exclusive distribution notice establishes no results reuse grant; dataset Apache licence is not a results grant.