Skip to content

Benchmark

Swallow v2 Japanese post-trained: GPQA

Science benchmarks measure how well models answer science questions. Higher scores are better.

Models measured
41
Scores recorded
41
Sources
1
IQ weight
1.25

Score distribution

0.565
Mean
0.592
Median
0.184
Std dev
0.261
10th percentile
0.786
90th percentile
0.246
Min
0.2460.853
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.25 for chat models, measured over 41 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1GPT 5.4reasoning on (medium)· Swallow LLM Leaderboard v2 data docs0.853
2Qwen3.5 122B-A10Breasoning on· Swallow LLM Leaderboard v2 data docs0.801
3Gemma 4 31B ITreasoning on· Swallow LLM Leaderboard v2 data docs0.795
4Qwen3.5 27Breasoning on· Swallow LLM Leaderboard v2 data docs0.795
5GPT 5reasoning on (medium)· Swallow LLM Leaderboard v2 data docs0.786
6Qwen3.5 35B A3Breasoning on· Swallow LLM Leaderboard v2 data docs0.775
7OpenAI o3reasoning on (medium)· Swallow LLM Leaderboard v2 data docs0.766
8GPT-5 minireasoning on (medium)· Swallow LLM Leaderboard v2 data docs0.75
9Gemma 4 26B A4Breasoning on· Swallow LLM Leaderboard v2 data docs0.739
10Nemotron 3 Super 120B A12Breasoning on· Swallow LLM Leaderboard v2 data docs0.739
11Qwen 3 235B A22B Thinking 2507reasoning on· Swallow LLM Leaderboard v2 data docs0.728
12Qwen3.5 9Breasoning on· Swallow LLM Leaderboard v2 data docs0.725
13Qwen3 Next 80B A3Breasoning on· Swallow LLM Leaderboard v2 data docs0.705
14Qwen3 235B A22B Instruct 2507· Swallow LLM Leaderboard v2 data docs0.701
15Qwen3.5 4Breasoning on· Swallow LLM Leaderboard v2 data docs0.688
16o3 Minireasoning on (medium)· Swallow LLM Leaderboard v2 data docs0.685
17GPT OSS 120Breasoning on (medium)· Swallow LLM Leaderboard v2 data docs0.658
18Qwen3 Next 80B A3B Instruct· Swallow LLM Leaderboard v2 data docs0.614
19Qwen3 32Breasoning on· Swallow LLM Leaderboard v2 data docs0.609
20GPT 4.1· Swallow LLM Leaderboard v2 data docs0.603
21Qwen3 30B A3Breasoning on· Swallow LLM Leaderboard v2 data docs0.592
22GPT OSS 20breasoning on (medium)· Swallow LLM Leaderboard v2 data docs0.587
23Qwen3 14Breasoning on· Swallow LLM Leaderboard v2 data docs0.587
24Llama 4 Scout· Swallow LLM Leaderboard v2 data docs0.54
25Gemma 4 E4B-itreasoning on· Swallow LLM Leaderboard v2 data docs0.511
26Olmo 3 32B Thinkreasoning on· Swallow LLM Leaderboard v2 data docs0.504
27Qwen3 8Breasoning on· Swallow LLM Leaderboard v2 data docs0.498
28Qwen3 4Breasoning on· Swallow LLM Leaderboard v2 data docs0.475
29GPT 4o· Swallow LLM Leaderboard v2 data docs0.453
30Llama 3.3 70B Instruct· Swallow LLM Leaderboard v2 data docs0.453
31Gemma 3 27B IT· Swallow LLM Leaderboard v2 data docs0.417
32Gemma 4 E2B-itreasoning on· Swallow LLM Leaderboard v2 data docs0.386
33Gemma 3 12B IT· Swallow LLM Leaderboard v2 data docs0.373
34Olmo 3 7B Thinkreasoning on· Swallow LLM Leaderboard v2 data docs0.371
35Qwen3 1.7Breasoning on· Swallow LLM Leaderboard v2 data docs0.326
36Qwen3.5 2Breasoning on· Swallow LLM Leaderboard v2 data docs0.324
37Llama 3.1 8B Instruct· Swallow LLM Leaderboard v2 data docs0.261
38Qwen3.5 0.8Breasoning on· Swallow LLM Leaderboard v2 data docs0.261
39Gemma 3 1b it· Swallow LLM Leaderboard v2 data docs0.248
40Qwen3 0.6Breasoning on· Swallow LLM Leaderboard v2 data docs0.248
41Gemma 3 4B IT· Swallow LLM Leaderboard v2 data docs0.246

Data credits: Swallow LLM Leaderboard v2, Swallow LLM team (Institute of Science Tokyo). Contents CC BY 4.0; Japanese post-trained task scores on the source's 0..1 scale. (licence)