Skip to content

Benchmark

Nejumi 4: AVG_swebench

Coding benchmarks compare models on programming tasks. Higher scores are better.

Models measured
114
Scores recorded
126
Sources
1
IQ weight
1.11

Score distribution

0.374
Mean
0.405
Median
0.206
Std dev
0.088
10th percentile
0.613
90th percentile
0
Min
00.8
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.11 for chat models, measured over 114 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
101Qwen: Qwen3 VL 8B ThinkingQwen/Qwen3-VL-8B-Thinking· Nejumi Leaderboard 4 data docs0.15
102Qwen3 30B A3BQwen/Qwen3-30B-A3B: reasoning-enabled· Nejumi Leaderboard 4 data docs0.15
103GPT-4.1 nanoopenai/gpt-4-1-nano-2025-04-14· Nejumi Leaderboard 4 data docs0.132
104Mistral Largemistral/mistral-large-2411· Nejumi Leaderboard 4 data docs0.125
105Qwen 3 235B A22BQwen/Qwen3-235B-A22B: reasoning-enabled· Nejumi Leaderboard 4 data docs0.125
106Llama 4 Scoutmeta-llama/Llama-4-Scout-17B-16E-Instruct· Nejumi Leaderboard 4 data docs0.113
107Mistral Smallmistral/mistral-small-2503· Nejumi Leaderboard 4 data docs0.113
108Qwen3-4B-Thinking-2507Qwen/Qwen3-4B-Thinking-2507· Nejumi Leaderboard 4 data docs0.113
109Qwen3-4B-Thinking-2507Qwen/Qwen3-4B-Thinking-2507: reasoning-enabled· Nejumi Leaderboard 4 data docs0.113
110Ministral 3 8Bmistralai/Ministral-3-8B-Reasoning-2512· Nejumi Leaderboard 4 data docs0.1
111Nova Lite 1.0us.amazon.nova-lite-v1:0· Nejumi Leaderboard 4 data docs0.1
112Gemini 2.5 Flash-Litegoogle/gemini-2.5-flash-lite· Nejumi Leaderboard 4 data docs0.0875
113Ministral 3 14Bmistralai/Ministral-3-14B-Reasoning-2512· Nejumi Leaderboard 4 data docs0.0875
114Qwen3.5 4BQwen/Qwen3.5-4B: reasoning-enabled· Nejumi Leaderboard 4 data docs0.0875
115Gemma 3 27B ITgoogle/gemma-3-27b-it· Nejumi Leaderboard 4 data docs0.075
116Qwen3 4BQwen/Qwen3-4B: reasoning-enabled· Nejumi Leaderboard 4 data docs0.075
117Qwen3 8BQwen/Qwen3-8B: reasoning-enabled· Nejumi Leaderboard 4 data docs0.075
118GPT-4o miniopenai/gpt-4o-mini-2024-07-18· Nejumi Leaderboard 4 data docs0.0625
119ERNIE 4.5 21B A3Bbaidu/ERNIE-4.5-21B-A3B-Thinking· Nejumi Leaderboard 4 data docs0.05
120Nova Micro 1.0us.amazon.nova-micro-v1:0· Nejumi Leaderboard 4 data docs0.05
121Ministral 3 3Bmistralai/Ministral-3-3B-Reasoning-2512· Nejumi Leaderboard 4 data docs0.0375
122Qwen3 1.7BQwen/Qwen3-1.7B: reasoning-enabled· Nejumi Leaderboard 4 data docs0.0375
123Gemma 3 12B ITgoogle/gemma-3-12b-it· Nejumi Leaderboard 4 data docs0.025
124Gemma 3 4B ITgoogle/gemma-3-4b-it· Nejumi Leaderboard 4 data docs0
125Llama 3.2 3Bmeta-llama/Llama-3.2-3B-Instruct· Nejumi Leaderboard 4 data docs0
126Qwen3 0.6BQwen/Qwen3-0.6B: reasoning-enabled· Nejumi Leaderboard 4 data docs0

Showing 101–126 of 126 results.

Data credits: Nejumi Leaderboard 4, Weights & Biases Japan; public publisher-selected numeric evaluation results