Skip to content

Benchmark

Swallow v2 Japanese post-trained: JamC-QA

Higher scores are better.

Models measured
41
Scores recorded
41
Sources
1
IQ weight
1.29

Score distribution

0.508
Mean
0.488
Median
0.178
Std dev
0.285
10th percentile
0.747
90th percentile
0.239
Min
0.2390.891
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.29 for chat models, measured over 41 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1GPT 5.4reasoning on (medium)· Swallow LLM Leaderboard v2 data docs0.891
2GPT 5reasoning on (medium)· Swallow LLM Leaderboard v2 data docs0.858
3OpenAI o3reasoning on (medium)· Swallow LLM Leaderboard v2 data docs0.851
4GPT 4.1· Swallow LLM Leaderboard v2 data docs0.79
5GPT 4o· Swallow LLM Leaderboard v2 data docs0.747
6GPT-5 minireasoning on (medium)· Swallow LLM Leaderboard v2 data docs0.701
7Gemma 4 31B ITreasoning on· Swallow LLM Leaderboard v2 data docs0.686
8Qwen3.5 122B-A10Breasoning on· Swallow LLM Leaderboard v2 data docs0.671
9Gemma 4 26B A4Breasoning on· Swallow LLM Leaderboard v2 data docs0.647
10Qwen 3 235B A22B Thinking 2507reasoning on· Swallow LLM Leaderboard v2 data docs0.641
11Qwen3 235B A22B Instruct 2507· Swallow LLM Leaderboard v2 data docs0.636
12Qwen3 Next 80B A3Breasoning on· Swallow LLM Leaderboard v2 data docs0.614
13Qwen3.5 35B A3Breasoning on· Swallow LLM Leaderboard v2 data docs0.611
14Qwen3 Next 80B A3B Instruct· Swallow LLM Leaderboard v2 data docs0.599
15Nemotron 3 Super 120B A12Breasoning on· Swallow LLM Leaderboard v2 data docs0.593
16Qwen3.5 27Breasoning on· Swallow LLM Leaderboard v2 data docs0.591
17Llama 4 Scout· Swallow LLM Leaderboard v2 data docs0.579
18GPT OSS 120Breasoning on (medium)· Swallow LLM Leaderboard v2 data docs0.518
19o3 Minireasoning on (medium)· Swallow LLM Leaderboard v2 data docs0.507
20Qwen3.5 9Breasoning on· Swallow LLM Leaderboard v2 data docs0.489
21Gemma 3 27B IT· Swallow LLM Leaderboard v2 data docs0.488
22Llama 3.3 70B Instruct· Swallow LLM Leaderboard v2 data docs0.484
23Qwen3 32Breasoning on· Swallow LLM Leaderboard v2 data docs0.472
24Qwen3 30B A3Breasoning on· Swallow LLM Leaderboard v2 data docs0.455
25Qwen3 14Breasoning on· Swallow LLM Leaderboard v2 data docs0.444
26Gemma 4 E4B-itreasoning on· Swallow LLM Leaderboard v2 data docs0.424
27GPT OSS 20breasoning on (medium)· Swallow LLM Leaderboard v2 data docs0.405
28Gemma 3 12B IT· Swallow LLM Leaderboard v2 data docs0.401
29Qwen3 8Breasoning on· Swallow LLM Leaderboard v2 data docs0.401
30Qwen3.5 4Breasoning on· Swallow LLM Leaderboard v2 data docs0.395
31Gemma 4 E2B-itreasoning on· Swallow LLM Leaderboard v2 data docs0.361
32Qwen3 4Breasoning on· Swallow LLM Leaderboard v2 data docs0.356
33Olmo 3 32B Thinkreasoning on· Swallow LLM Leaderboard v2 data docs0.333
34Llama 3.1 8B Instruct· Swallow LLM Leaderboard v2 data docs0.31
35Qwen3 1.7Breasoning on· Swallow LLM Leaderboard v2 data docs0.292
36Olmo 3 7B Thinkreasoning on· Swallow LLM Leaderboard v2 data docs0.286
37Gemma 3 4B IT· Swallow LLM Leaderboard v2 data docs0.285
38Qwen3.5 2Breasoning on· Swallow LLM Leaderboard v2 data docs0.283
39Gemma 3 1b it· Swallow LLM Leaderboard v2 data docs0.249
40Qwen3.5 0.8Breasoning on· Swallow LLM Leaderboard v2 data docs0.245
41Qwen3 0.6Breasoning on· Swallow LLM Leaderboard v2 data docs0.239

Data credits: Swallow LLM Leaderboard v2, Swallow LLM team (Institute of Science Tokyo). Contents CC BY 4.0; Japanese post-trained task scores on the source's 0..1 scale. (licence)