Skip to content

Benchmark

Swallow v2 Japanese post-trained: WMT20enja

Multilingual benchmarks compare model performance across languages. Higher scores are better.

Models measured
41
Scores recorded
41
Sources
1
IQ weight
2.02

Score distribution

0.206
Mean
0.24
Median
0.082
Std dev
0.08
10th percentile
0.276
90th percentile
0
Min
00.285
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 2.02 for chat models, measured over 41 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Gemma 4 31B ITreasoning on· Swallow LLM Leaderboard v2 data docs0.285
2GPT 4o· Swallow LLM Leaderboard v2 data docs0.282
3Gemma 4 26B A4Breasoning on· Swallow LLM Leaderboard v2 data docs0.28
4GPT 4.1· Swallow LLM Leaderboard v2 data docs0.278
5OpenAI o3reasoning on (medium)· Swallow LLM Leaderboard v2 data docs0.276
6GPT 5reasoning on (medium)· Swallow LLM Leaderboard v2 data docs0.272
7GPT 5.4reasoning on (medium)· Swallow LLM Leaderboard v2 data docs0.272
8GPT-5 minireasoning on (medium)· Swallow LLM Leaderboard v2 data docs0.267
9Qwen3.5 122B-A10Breasoning on· Swallow LLM Leaderboard v2 data docs0.263
10GPT OSS 120Breasoning on (medium)· Swallow LLM Leaderboard v2 data docs0.262
11Gemma 4 E4B-itreasoning on· Swallow LLM Leaderboard v2 data docs0.261
12Qwen3 235B A22B Instruct 2507· Swallow LLM Leaderboard v2 data docs0.258
13Gemma 4 E2B-itreasoning on· Swallow LLM Leaderboard v2 data docs0.256
14Qwen 3 235B A22B Thinking 2507reasoning on· Swallow LLM Leaderboard v2 data docs0.255
15Qwen3.5 27Breasoning on· Swallow LLM Leaderboard v2 data docs0.252
16Gemma 3 27B IT· Swallow LLM Leaderboard v2 data docs0.25
17Qwen3.5 35B A3Breasoning on· Swallow LLM Leaderboard v2 data docs0.25
18Llama 3.3 70B Instruct· Swallow LLM Leaderboard v2 data docs0.245
19Nemotron 3 Super 120B A12Breasoning on· Swallow LLM Leaderboard v2 data docs0.244
20o3 Minireasoning on (medium)· Swallow LLM Leaderboard v2 data docs0.243
21Qwen3 Next 80B A3Breasoning on· Swallow LLM Leaderboard v2 data docs0.24
22Qwen3 Next 80B A3B Instruct· Swallow LLM Leaderboard v2 data docs0.24
23Llama 4 Scout· Swallow LLM Leaderboard v2 data docs0.237
24GPT OSS 20breasoning on (medium)· Swallow LLM Leaderboard v2 data docs0.234
25Qwen3 14Breasoning on· Swallow LLM Leaderboard v2 data docs0.228
26Gemma 3 12B IT· Swallow LLM Leaderboard v2 data docs0.225
27Qwen3 32Breasoning on· Swallow LLM Leaderboard v2 data docs0.218
28Qwen3 8Breasoning on· Swallow LLM Leaderboard v2 data docs0.202
29Llama 3.1 8B Instruct· Swallow LLM Leaderboard v2 data docs0.187
30Gemma 3 4B IT· Swallow LLM Leaderboard v2 data docs0.186
31Qwen3 30B A3Breasoning on· Swallow LLM Leaderboard v2 data docs0.177
32Qwen3.5 9Breasoning on· Swallow LLM Leaderboard v2 data docs0.174
33Olmo 3 32B Thinkreasoning on· Swallow LLM Leaderboard v2 data docs0.172
34Qwen3 4Breasoning on· Swallow LLM Leaderboard v2 data docs0.149
35Olmo 3 7B Thinkreasoning on· Swallow LLM Leaderboard v2 data docs0.125
36Qwen3 1.7Breasoning on· Swallow LLM Leaderboard v2 data docs0.125
37Qwen3.5 4Breasoning on· Swallow LLM Leaderboard v2 data docs0.08
38Gemma 3 1b it· Swallow LLM Leaderboard v2 data docs0.004
39Qwen3 0.6Breasoning on· Swallow LLM Leaderboard v2 data docs0.001
40Qwen3.5 0.8Breasoning on· Swallow LLM Leaderboard v2 data docs0
41Qwen3.5 2Breasoning on· Swallow LLM Leaderboard v2 data docs0

Data credits: Swallow LLM Leaderboard v2, Swallow LLM team (Institute of Science Tokyo). Contents CC BY 4.0; Japanese post-trained task scores on the source's 0..1 scale. (licence)