Skip to content

Benchmark

Swallow v2 Japanese post-trained: PolyMathJaHT

Math benchmarks measure how well models solve maths problems. Higher scores are better.

Models measured
41
Scores recorded
41
Sources
1
IQ weight
1.21

Score distribution

0.351
Mean
0.404
Median
0.215
Std dev
0.032
10th percentile
0.592
90th percentile
0.008
Min
0.0080.776
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.21 for chat models, measured over 41 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1GPT 5.4reasoning on (medium)· Swallow LLM Leaderboard v2 data docs0.776
2Gemma 4 31B ITreasoning on· Swallow LLM Leaderboard v2 data docs0.684
3GPT 5reasoning on (medium)· Swallow LLM Leaderboard v2 data docs0.624
4Gemma 4 26B A4Breasoning on· Swallow LLM Leaderboard v2 data docs0.6
5Qwen3.5 27Breasoning on· Swallow LLM Leaderboard v2 data docs0.592
6Qwen3.5 122B-A10Breasoning on· Swallow LLM Leaderboard v2 data docs0.588
7Qwen 3 235B A22B Thinking 2507reasoning on· Swallow LLM Leaderboard v2 data docs0.576
8OpenAI o3reasoning on (medium)· Swallow LLM Leaderboard v2 data docs0.548
9Qwen3.5 35B A3Breasoning on· Swallow LLM Leaderboard v2 data docs0.548
10Nemotron 3 Super 120B A12Breasoning on· Swallow LLM Leaderboard v2 data docs0.544
11Qwen3 Next 80B A3Breasoning on· Swallow LLM Leaderboard v2 data docs0.54
12GPT-5 minireasoning on (medium)· Swallow LLM Leaderboard v2 data docs0.512
13GPT OSS 120Breasoning on (medium)· Swallow LLM Leaderboard v2 data docs0.464
14Qwen3.5 9Breasoning on· Swallow LLM Leaderboard v2 data docs0.46
15Olmo 3 32B Thinkreasoning on· Swallow LLM Leaderboard v2 data docs0.452
16o3 Minireasoning on (medium)· Swallow LLM Leaderboard v2 data docs0.444
17Qwen3 14Breasoning on· Swallow LLM Leaderboard v2 data docs0.44
18Qwen3 32Breasoning on· Swallow LLM Leaderboard v2 data docs0.44
19Qwen3 30B A3Breasoning on· Swallow LLM Leaderboard v2 data docs0.416
20Qwen3 8Breasoning on· Swallow LLM Leaderboard v2 data docs0.412
21Qwen3.5 4Breasoning on· Swallow LLM Leaderboard v2 data docs0.404
22GPT OSS 20breasoning on (medium)· Swallow LLM Leaderboard v2 data docs0.372
23Qwen3 Next 80B A3B Instruct· Swallow LLM Leaderboard v2 data docs0.372
24Olmo 3 7B Thinkreasoning on· Swallow LLM Leaderboard v2 data docs0.368
25Qwen3 235B A22B Instruct 2507· Swallow LLM Leaderboard v2 data docs0.368
26Qwen3 4Breasoning on· Swallow LLM Leaderboard v2 data docs0.364
27Gemma 4 E4B-itreasoning on· Swallow LLM Leaderboard v2 data docs0.26
28Qwen3 1.7Breasoning on· Swallow LLM Leaderboard v2 data docs0.244
29Gemma 4 E2B-itreasoning on· Swallow LLM Leaderboard v2 data docs0.168
30GPT 4.1· Swallow LLM Leaderboard v2 data docs0.148
31Llama 4 Scout· Swallow LLM Leaderboard v2 data docs0.148
32Gemma 3 27B IT· Swallow LLM Leaderboard v2 data docs0.104
33Gemma 3 12B IT· Swallow LLM Leaderboard v2 data docs0.1
34GPT 4o· Swallow LLM Leaderboard v2 data docs0.084
35Llama 3.3 70B Instruct· Swallow LLM Leaderboard v2 data docs0.08
36Qwen3 0.6Breasoning on· Swallow LLM Leaderboard v2 data docs0.048
37Qwen3.5 2Breasoning on· Swallow LLM Leaderboard v2 data docs0.032
38Gemma 3 4B IT· Swallow LLM Leaderboard v2 data docs0.02
39Llama 3.1 8B Instruct· Swallow LLM Leaderboard v2 data docs0.016
40Gemma 3 1b it· Swallow LLM Leaderboard v2 data docs0.012
41Qwen3.5 0.8Breasoning on· Swallow LLM Leaderboard v2 data docs0.008

Data credits: Swallow LLM Leaderboard v2, Swallow LLM team (Institute of Science Tokyo). Contents CC BY 4.0; Japanese post-trained task scores on the source's 0..1 scale. (licence)