Skip to content

Benchmark

Nejumi 4: AVG_swebench

Coding benchmarks compare models on programming tasks. Higher scores are better.

Models measured
114
Scores recorded
126
Sources
1
IQ weight
1.11

Score distribution

0.374
Mean
0.405
Median
0.206
Std dev
0.088
10th percentile
0.613
90th percentile
0
Min
00.8
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.11 for chat models, measured over 114 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Claude Fable 5anthropic/claude-fable-5: adaptive-thinking-max with fallback to Opus 4.8· Nejumi Leaderboard 4 data docs0.8
2Claude Opus 5anthropic/claude-opus-5: adaptive-thinking-max· Nejumi Leaderboard 4 data docs0.75
3GPT-5.6 Solopenai/gpt-5.6-sol: max-effort· Nejumi Leaderboard 4 data docs0.738
4GPT 5.5openai/gpt-5.5-2026-04-23: high-effort· Nejumi Leaderboard 4 data docs0.713
5GPT-5.6 Terraopenai/gpt-5.6-terra: max-effort· Nejumi Leaderboard 4 data docs0.7
6Claude Opus 4.8anthropic/claude-opus-4.8: adaptive-thinking-xhigh· Nejumi Leaderboard 4 data docs0.688
7GPT 5.5openai/gpt-5.5-2026-04-23: xhigh-effort· Nejumi Leaderboard 4 data docs0.688
8Gemini 3 Flashgoogle/gemini-3-flash-preview· Nejumi Leaderboard 4 data docs0.663
9Grok 4.5xai/grok-4.5: reasoning-high· Nejumi Leaderboard 4 data docs0.663
10Claude Opus 4.7anthropic/claude-opus-4.7: adaptive-thinking-xhigh· Nejumi Leaderboard 4 data docs0.65
11Kimi K3moonshotai/kimi-k3: reasoning-max· Nejumi Leaderboard 4 data docs0.638
12Gemini 3.1 Pro Previewgoogle/gemini-3.1-pro-preview· Nejumi Leaderboard 4 data docs0.625
13Claude Sonnet 4.5anthropic/claude-sonnet-4-5-20250929: extended-thinking· Nejumi Leaderboard 4 data docs0.613
14GLM 5.2zai-org/GLM-5.2: reasoning-max· Nejumi Leaderboard 4 data docs0.613
15Claude Opus 4.1anthropic/claude-opus-4-1-20250805: extended-thinking· Nejumi Leaderboard 4 data docs0.6
16Claude Opus 4.5anthropic/claude-opus-4.5-20251125: extended-thinking· Nejumi Leaderboard 4 data docs0.6
17GLM 5.2z-ai/glm-5.2: openrouter-reasoning-xhigh· Nejumi Leaderboard 4 data docs0.6
18Claude Opus 4.5anthropic/claude-opus-4.5-20251125: no-thinking· Nejumi Leaderboard 4 data docs0.588
19Claude Sonnet 4.6anthropic/claude-sonnet-4.6: extended-thinking· Nejumi Leaderboard 4 data docs0.588
20Gemini 2.5 Progoogle/gemini-2.5-pro· Nejumi Leaderboard 4 data docs0.588
21GPT 5openai/gpt-5-2025-08-07: high-effort· Nejumi Leaderboard 4 data docs0.588
22Grok 4.20xai/grok-4.20-0309-reasoning· Nejumi Leaderboard 4 data docs0.588
23Qwen3.6 Max Previewqwen/qwen3.6-max-preview: openrouter-reasoning· Nejumi Leaderboard 4 data docs0.588
24DeepSeek V4 Flashdeepseek-ai/DeepSeek-V4-Flash: reasoning-max· Nejumi Leaderboard 4 data docs0.575
25DeepSeek V4 Prodeepseek/deepseek-v4-pro: thinking-max· Nejumi Leaderboard 4 data docs0.575
26GLM 4.7zai-org/GLM-4.7: reasoning-enabled· Nejumi Leaderboard 4 data docs0.575
27GPT-5.6 Lunaopenai/gpt-5.6-luna: max-effort· Nejumi Leaderboard 4 data docs0.575
28Kimi K2.5moonshotai/Kimi-K2.5: reasoning-enabled· Nejumi Leaderboard 4 data docs0.575
29Claude Opus 4.6anthropic/claude-opus-4-6: extended-thinking· Nejumi Leaderboard 4 data docs0.563
30Gemini 3.5 Flashgoogle/gemini-3.5-flash· Nejumi Leaderboard 4 data docs0.563
31Claude Haiku 4.5anthropic/claude-haiku-4-5-20251001: extended-thinking· Nejumi Leaderboard 4 data docs0.55
32GLM 5zai-org/GLM-5: reasoning-enabled· Nejumi Leaderboard 4 data docs0.55
33GPT 5.2openai/gpt-5.2-2025-12-11: xhigh-effort· Nejumi Leaderboard 4 data docs0.55
34DeepSeek V4 Prodeepseek-ai/DeepSeek-V4-Pro: reasoning-max· Nejumi Leaderboard 4 data docs0.538
35Gemma 4 31B ITgoogle/gemma-4-31b-it· Nejumi Leaderboard 4 data docs0.538
36Claude Sonnet 3.7anthropic/claude-3.7-sonnet-20250219: extended-thinking· Nejumi Leaderboard 4 data docs0.525
37GPT 5.4openai/gpt-5.4-2026-03-05: high-effort· Nejumi Leaderboard 4 data docs0.525
38Claude Sonnet 4anthropic/claude-sonnet-4-20250514: extended-thinking· Nejumi Leaderboard 4 data docs0.513
39Gemini 3.6 Flashgoogle/gemini-3.6-flash· Nejumi Leaderboard 4 data docs0.513
40GPT 5.1openai/gpt-5.1-2025-11-13: high-effort· Nejumi Leaderboard 4 data docs0.513
41Qwen3.5 397B A17BQwen/Qwen3.5-397B-A17B: reasoning-enabled· Nejumi Leaderboard 4 data docs0.513
42Qwen3.6 27BQwen/Qwen3.6-27B: reasoning-enabled· Nejumi Leaderboard 4 data docs0.513
43Gemma 4 26B A4Bgoogle/gemma-4-26B-A4B-it· Nejumi Leaderboard 4 data docs0.5
44Kimi K2 Thinkingmoonshotai/kimi-k2-thinking· Nejumi Leaderboard 4 data docs0.5
45MiniMax M3MiniMaxAI/MiniMax-M3: reasoning-enabled· Nejumi Leaderboard 4 data docs0.5
46Claude Opus 4anthropic/claude-opus-4-20250514: no-thinking· Nejumi Leaderboard 4 data docs0.488
47DeepSeek R1deepseek-ai/DeepSeek-R1-0528: reasoning-enabled· Nejumi Leaderboard 4 data docs0.488
48DeepSeek Reasonerdeepseek/DeepSeek-V3.2 (Thinking Mode)· Nejumi Leaderboard 4 data docs0.488
49GPT-5.4 miniopenai/gpt-5.4-mini-2026-03-17: high-effort· Nejumi Leaderboard 4 data docs0.488
50Claude Sonnet 4anthropic/claude-sonnet-4-20250514: no-thinking· Nejumi Leaderboard 4 data docs0.481

Showing 1–50 of 126 results.

Data credits: Nejumi Leaderboard 4, Weights & Biases Japan; public publisher-selected numeric evaluation results