Skip to content

Benchmark

Swallow v2 Japanese post-trained: WMT20jaen

Multilingual benchmarks compare model performance across languages. Higher scores are better.

Models measured
41
Scores recorded
41
Sources
1
IQ weight
2.00

Score distribution

0.2
Mean
0.221
Median
0.066
Std dev
0.141
10th percentile
0.249
90th percentile
0
Min
00.265
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 2.00 for chat models, measured over 41 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1GPT 4o· Swallow LLM Leaderboard v2 data docs0.265
2Gemma 4 26B A4Breasoning on· Swallow LLM Leaderboard v2 data docs0.261
3Gemma 4 31B ITreasoning on· Swallow LLM Leaderboard v2 data docs0.26
4GPT 4.1· Swallow LLM Leaderboard v2 data docs0.26
5GPT 5.4reasoning on (medium)· Swallow LLM Leaderboard v2 data docs0.249
6Qwen3.5 122B-A10Breasoning on· Swallow LLM Leaderboard v2 data docs0.247
7Llama 3.3 70B Instruct· Swallow LLM Leaderboard v2 data docs0.246
8Qwen3.5 27Breasoning on· Swallow LLM Leaderboard v2 data docs0.241
9GPT-5 minireasoning on (medium)· Swallow LLM Leaderboard v2 data docs0.239
10Qwen3.5 35B A3Breasoning on· Swallow LLM Leaderboard v2 data docs0.239
11Gemma 3 27B IT· Swallow LLM Leaderboard v2 data docs0.238
12GPT 5reasoning on (medium)· Swallow LLM Leaderboard v2 data docs0.236
13Llama 4 Scout· Swallow LLM Leaderboard v2 data docs0.23
14Qwen3 235B A22B Instruct 2507· Swallow LLM Leaderboard v2 data docs0.23
15Gemma 3 12B IT· Swallow LLM Leaderboard v2 data docs0.229
16Qwen 3 235B A22B Thinking 2507reasoning on· Swallow LLM Leaderboard v2 data docs0.229
17Gemma 4 E4B-itreasoning on· Swallow LLM Leaderboard v2 data docs0.228
18Qwen3 Next 80B A3B Instruct· Swallow LLM Leaderboard v2 data docs0.228
19Gemma 4 E2B-itreasoning on· Swallow LLM Leaderboard v2 data docs0.227
20Nemotron 3 Super 120B A12Breasoning on· Swallow LLM Leaderboard v2 data docs0.222
21o3 Minireasoning on (medium)· Swallow LLM Leaderboard v2 data docs0.221
22Qwen3 14Breasoning on· Swallow LLM Leaderboard v2 data docs0.219
23OpenAI o3reasoning on (medium)· Swallow LLM Leaderboard v2 data docs0.216
24Qwen3 32Breasoning on· Swallow LLM Leaderboard v2 data docs0.216
25Qwen3 30B A3Breasoning on· Swallow LLM Leaderboard v2 data docs0.215
26Qwen3.5 9Breasoning on· Swallow LLM Leaderboard v2 data docs0.209
27GPT OSS 120Breasoning on (medium)· Swallow LLM Leaderboard v2 data docs0.208
28GPT OSS 20breasoning on (medium)· Swallow LLM Leaderboard v2 data docs0.205
29Qwen3 8Breasoning on· Swallow LLM Leaderboard v2 data docs0.201
30Qwen3 Next 80B A3Breasoning on· Swallow LLM Leaderboard v2 data docs0.195
31Llama 3.1 8B Instruct· Swallow LLM Leaderboard v2 data docs0.194
32Gemma 3 4B IT· Swallow LLM Leaderboard v2 data docs0.189
33Qwen3 4Breasoning on· Swallow LLM Leaderboard v2 data docs0.189
34Olmo 3 32B Thinkreasoning on· Swallow LLM Leaderboard v2 data docs0.185
35Qwen3 1.7Breasoning on· Swallow LLM Leaderboard v2 data docs0.151
36Qwen3.5 4Breasoning on· Swallow LLM Leaderboard v2 data docs0.147
37Olmo 3 7B Thinkreasoning on· Swallow LLM Leaderboard v2 data docs0.141
38Gemma 3 1b it· Swallow LLM Leaderboard v2 data docs0.083
39Qwen3.5 2Breasoning on· Swallow LLM Leaderboard v2 data docs0.002
40Qwen3 0.6Breasoning on· Swallow LLM Leaderboard v2 data docs0
41Qwen3.5 0.8Breasoning on· Swallow LLM Leaderboard v2 data docs0

Data credits: Swallow LLM Leaderboard v2, Swallow LLM team (Institute of Science Tokyo). Contents CC BY 4.0; Japanese post-trained task scores on the source's 0..1 scale. (licence)