Skip to content

Benchmark

Nejumi 4: AVG_mtbench

People compare model answers and choose which they prefer. Higher scores are better.

Models measured
114
Scores recorded
126
Sources
1
IQ weight
2.03

Score distribution

0.908
Mean
0.942
Median
0.103
Std dev
0.826
10th percentile
0.976
90th percentile
0.396
Min
0.3960.988
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 2.03 for chat models, measured over 114 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
51Qwen3.5 122B-A10BQwen/Qwen3.5-122B-A10B: reasoning-enabled· Nejumi Leaderboard 4 data docs0.951
52GPT-4.1 miniopenai/gpt-4-1-mini-2025-04-14· Nejumi Leaderboard 4 data docs0.95
53GPT 5.4openai/gpt-5.4-2026-03-05: high-effort· Nejumi Leaderboard 4 data docs0.949
54GPT 5.5openai/gpt-5.5-2026-04-23: xhigh-effort· Nejumi Leaderboard 4 data docs0.949
55Claude Fable 5anthropic/claude-fable-5: adaptive-thinking-max with fallback to Opus 4.8· Nejumi Leaderboard 4 data docs0.948
56DeepSeek V3.2deepseek-ai/DeepSeek-V3.2: reasoning-enabled· Nejumi Leaderboard 4 data docs0.948
57Claude Sonnet 4anthropic/claude-sonnet-4-20250514: extended-thinking· Nejumi Leaderboard 4 data docs0.946
58Claude Sonnet 3.7anthropic/claude-3.7-sonnet-20250219: no-thinking· Nejumi Leaderboard 4 data docs0.944
59Claude Sonnet 4anthropic/claude-sonnet-4-20250514: no-thinking· Nejumi Leaderboard 4 data docs0.943
60Gemma 4 E4B-itgoogle/gemma-4-E4B-it· Nejumi Leaderboard 4 data docs0.943
61GPT 5openai/gpt-5-2025-08-07: high-effort· Nejumi Leaderboard 4 data docs0.943
62Claude Opus 4anthropic/claude-opus-4-20250514: no-thinking· Nejumi Leaderboard 4 data docs0.942
63GPT-5.4 miniopenai/gpt-5.4-mini-2026-03-17: high-effort· Nejumi Leaderboard 4 data docs0.942
64Claude Opus 4.5anthropic/claude-opus-4.5-20251125: no-thinking· Nejumi Leaderboard 4 data docs0.941
65Grok 3 Minigrok-3-mini· Nejumi Leaderboard 4 data docs0.941
66Claude Opus 4.1anthropic/claude-opus-4-1-20250805: extended-thinking· Nejumi Leaderboard 4 data docs0.941
67DeepSeek V4 Flashdeepseek-ai/DeepSeek-V4-Flash: reasoning-max· Nejumi Leaderboard 4 data docs0.94
68GPT-5 nanoopenai/gpt-5-nano-2025-08-07: high-effort· Nejumi Leaderboard 4 data docs0.94
69Grok 4.1 Fastx-ai/grok-4-1-fast-reasoning· Nejumi Leaderboard 4 data docs0.939
70QwQ 32BQwen/QwQ-32B: reasoning-enabled· Nejumi Leaderboard 4 data docs0.936
71Claude Haiku 4.5anthropic/claude-haiku-4-5-20251001: extended-thinking· Nejumi Leaderboard 4 data docs0.935
72Qwen3 14BQwen/Qwen3-14B: reasoning-enabled· Nejumi Leaderboard 4 data docs0.934
73Qwen3 30B A3BQwen/Qwen3-30B-A3B: reasoning-enabled· Nejumi Leaderboard 4 data docs0.934
74Grok 4grok-4· Nejumi Leaderboard 4 data docs0.934
75Qwen3.6 Max Previewqwen/qwen3.6-max-preview: openrouter-reasoning· Nejumi Leaderboard 4 data docs0.934
76Qwen3 VL 32B ThinkingQwen/Qwen3-VL-32B-Thinking· Nejumi Leaderboard 4 data docs0.933
77Claude Opus 4.5anthropic/claude-opus-4.5-20251125: extended-thinking· Nejumi Leaderboard 4 data docs0.933
78Claude Sonnet 3.7anthropic/claude-3.7-sonnet-20250219: extended-thinking· Nejumi Leaderboard 4 data docs0.931
79Claude Sonnet 4.5anthropic/claude-sonnet-4-5-20250929: extended-thinking· Nejumi Leaderboard 4 data docs0.931
80Grok 3grok-3· Nejumi Leaderboard 4 data docs0.931
81Qwen3 Next 80B A3BQwen/Qwen3-Next-80B-A3B-Thinking· Nejumi Leaderboard 4 data docs0.931
82Qwen3 32BQwen/Qwen3-32B: reasoning-enabled· Nejumi Leaderboard 4 data docs0.926
83Qwen3 Next 80B A3B InstructQwen/Qwen3-Next-80B-A3B-Instruct· Nejumi Leaderboard 4 data docs0.926
84DeepSeek V3deepseek-ai/DeepSeek-V3-0324· Nejumi Leaderboard 4 data docs0.926
85QwQ 32BQwen/QwQ-32B· Nejumi Leaderboard 4 data docs0.923
86Qwen3 Next 80B A3BQwen/Qwen3-Next-80B-A3B-Thinking: reasoning-enabled· Nejumi Leaderboard 4 data docs0.923
87Gemma 4 E2B-itgoogle/gemma-4-E2B-it· Nejumi Leaderboard 4 data docs0.922
88Qwen: Qwen3 VL 8B ThinkingQwen/Qwen3-VL-8B-Thinking· Nejumi Leaderboard 4 data docs0.922
89MiniMax M2.1MiniMaxAI/MiniMax-M2.1: reasoning-enabled· Nejumi Leaderboard 4 data docs0.92
90GLM 4.7 Flashzai-org/GLM-4.7-Flash: reasoning-enabled· Nejumi Leaderboard 4 data docs0.919
91Claude 3.5 Sonnetanthropic/claude-3.5-sonnet-20241022· Nejumi Leaderboard 4 data docs0.918
92Qwen3.5 9BQwen/Qwen3.5-9B: reasoning-enabled· Nejumi Leaderboard 4 data docs0.915
93Qwen3.5 4BQwen/Qwen3.5-4B: reasoning-enabled· Nejumi Leaderboard 4 data docs0.912
94MiniMax M2MiniMaxAI/MiniMax-M2: reasoning-enabled· Nejumi Leaderboard 4 data docs0.911
95Qwen3 30B A3BQwen/Qwen3-30B-A3B-Thinking-2507: reasoning-enabled· Nejumi Leaderboard 4 data docs0.909
96Qwen3 8BQwen/Qwen3-8B: reasoning-enabled· Nejumi Leaderboard 4 data docs0.906
97GPT 4oopenai/gpt-4o-2024-11-20· Nejumi Leaderboard 4 data docs0.9
98Mistral Large 3 675b Instructmistralai/Mistral-Large-3-675B-Instruct-2512· Nejumi Leaderboard 4 data docs0.891
99Gemma 3 27B ITgoogle/gemma-3-27b-it· Nejumi Leaderboard 4 data docs0.883
100Gemini 2.5 Flash-Litegoogle/gemini-2.5-flash-lite· Nejumi Leaderboard 4 data docs0.882

Showing 51–100 of 126 results.

Data credits: Nejumi Leaderboard 4, Weights & Biases Japan; public publisher-selected numeric evaluation results