Skip to content

Benchmark

Horangi 4 subset: swebench_verified_official_80

Coding benchmarks compare models on programming tasks. Higher scores are better.

Models measured
79
Scores recorded
83
Sources
1
IQ weight
1.68

Score distribution

0.41
Mean
0.475
Median
0.209
Std dev
0.063
10th percentile
0.654
90th percentile
0
Min
00.738
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.68 for chat models, measured over 79 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Gemini 3 Flashgemini-3-flash-preview· Horangi Korean Leaderboard 4 data docs0.738
2GPT 5.4gpt-5.4-2026-03-05: xhigh-effort· Horangi Korean Leaderboard 4 data docs0.738
3GPT 5.5gpt-5.5: xhigh-effort· Horangi Korean Leaderboard 4 data docs0.7
4Gemini 3 Pro Previewgemini-3-pro-preview: high-effort· Horangi Korean Leaderboard 4 data docs0.688
5Gemini 3.1 Pro Previewgemini-3.1-pro-preview: high-effort· Horangi Korean Leaderboard 4 data docs0.675
6GPT-5.4 minigpt-5.4-mini-2026-03-17: xhigh-effort· Horangi Korean Leaderboard 4 data docs0.675
7Claude Fable 5claude-fable-5: high-effort· Horangi Korean Leaderboard 4 data docs0.663
8Qwen3.7 Plusqwen3.7-plus· Horangi Korean Leaderboard 4 data docs0.662
9Grok 4.20grok-4.20· Horangi Korean Leaderboard 4 data docs0.654
10GPT-5.6 Solgpt-5.6-sol: max-effort· Horangi Korean Leaderboard 4 data docs0.65
11Gemini 3.5 Flashgemini-3.5-flash· Horangi Korean Leaderboard 4 data docs0.646
12Qwen3.7 Maxqwen3.7-max· Horangi Korean Leaderboard 4 data docs0.645
13Claude Opus 4.5claude-opus-4-5-20251101: high-effort· Horangi Korean Leaderboard 4 data docs0.628
14Grok 4.5grok-4.5· Horangi Korean Leaderboard 4 data docs0.595
15Claude Opus 4.6claude-opus-4-6: high-effort· Horangi Korean Leaderboard 4 data docs0.59
16Gemma 4 31B ITGemma-4-31B-IT-NVFP4: enable-thinking· Horangi Korean Leaderboard 4 data docs0.588
17GPT-5.6 Terragpt-5.6-terra: max-effort· Horangi Korean Leaderboard 4 data docs0.588
18Kimi K2.5kimi-k2.5· Horangi Korean Leaderboard 4 data docs0.584
19Claude Opus 4.8claude-opus-4-8: high-effort· Horangi Korean Leaderboard 4 data docs0.575
20Gemma 4 31B ITgemma-4-31B-it: enable-thinking· Horangi Korean Leaderboard 4 data docs0.575
21GLM 5.2glm-5.2: high-effort· Horangi Korean Leaderboard 4 data docs0.562
22Grok 4.20grok-4.20-beta· Horangi Korean Leaderboard 4 data docs0.551
23Claude Sonnet 4.6claude-sonnet-4-6: high-effort· Horangi Korean Leaderboard 4 data docs0.55
24Claude Sonnet 5claude-sonnet-5: high-effort· Horangi Korean Leaderboard 4 data docs0.55
25GPT 5gpt-5-2025-08-07· Horangi Korean Leaderboard 4 data docs0.55
26GLM 5 Turboglm-5-turbo· Horangi Korean Leaderboard 4 data docs0.544
27Claude Sonnet 4.5claude-sonnet-4-5-20250929: high-effort· Horangi Korean Leaderboard 4 data docs0.538
28GPT 5.4 Nanogpt-5.4-nano-2026-03-17: xhigh-effort· Horangi Korean Leaderboard 4 data docs0.538
29Qwen3.6 Plus Previewqwen3.6-plus-preview· Horangi Korean Leaderboard 4 data docs0.533
30Claude Opus 4.1claude-opus-4-1-20250805· Horangi Korean Leaderboard 4 data docs0.525
31DeepSeek Reasonerdeepseek-v3.2: high-effort· Horangi Korean Leaderboard 4 data docs0.513
32Qwen3.6 Flashqwen3.6-flash· Horangi Korean Leaderboard 4 data docs0.513
33MiMo V2 Promimo-v2-pro· Horangi Korean Leaderboard 4 data docs0.506
34GPT-5.6 Lunagpt-5.6-luna: max-effort· Horangi Korean Leaderboard 4 data docs0.5
35Qwen 3 235B A22B Thinking 2507qwen3-235b-a22b-thinking-2507· Horangi Korean Leaderboard 4 data docs0.5
36GLM 4.7glm-4.7· Horangi Korean Leaderboard 4 data docs0.494
37Claude Opus 4claude-opus-4-20250514: high-effort· Horangi Korean Leaderboard 4 data docs0.488
38MiniMax M2.7minimax-m2.7· Horangi Korean Leaderboard 4 data docs0.488
39Nemotron 3 Super 120B A12BNVIDIA-Nemotron-3-Super-120B-A12B-FP8· Horangi Korean Leaderboard 4 data docs0.488
40GLM 5glm-5· Horangi Korean Leaderboard 4 data docs0.481
41Claude Haiku 4.5claude-haiku-4-5-20251001: high-effort· Horangi Korean Leaderboard 4 data docs0.475
42GPT 5.1gpt-5.1-2025-11-13· Horangi Korean Leaderboard 4 data docs0.475
43Qwen3.5 397B A17Bqwen3.5-397b-a17b· Horangi Korean Leaderboard 4 data docs0.475
44GLM 5V Turboglm-5v-turbo· Horangi Korean Leaderboard 4 data docs0.474
45Gemini 3.1 Flash Litegemini-3.1-flash-lite-preview· Horangi Korean Leaderboard 4 data docs0.463
46Gemma 4 26B A4Bgemma-4-26B-A4B-it: enable-thinking· Horangi Korean Leaderboard 4 data docs0.45
47MiMo V2 Flashmimo-v2-flash· Horangi Korean Leaderboard 4 data docs0.45
48MiniMax M2.5minimax-m2.5· Horangi Korean Leaderboard 4 data docs0.449
49GPT 4.1gpt-4.1· Horangi Korean Leaderboard 4 data docs0.425
50Qwen3.5 122B-A10Bqwen3.5-122b-a10b· Horangi Korean Leaderboard 4 data docs0.413

Showing 1–50 of 83 results.

Data credits: Horangi 4, Weights & Biases Korea; public publisher-selected numeric evaluation results