Skip to content

Benchmark

Nejumi 4: AVG_mtbench

People compare model answers and choose which they prefer. Higher scores are better.

Models measured
114
Scores recorded
126
Sources
1
IQ weight
2.03

Score distribution

0.908
Mean
0.942
Median
0.103
Std dev
0.826
10th percentile
0.976
90th percentile
0.396
Min
0.3960.988
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 2.03 for chat models, measured over 114 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Claude Opus 5anthropic/claude-opus-5: adaptive-thinking-max· Nejumi Leaderboard 4 data docs0.988
2Claude Sonnet 4.6anthropic/claude-sonnet-4.6: extended-thinking· Nejumi Leaderboard 4 data docs0.984
3Kimi K3moonshotai/kimi-k3: reasoning-max· Nejumi Leaderboard 4 data docs0.984
4Gemini 3.6 Flashgoogle/gemini-3.6-flash· Nejumi Leaderboard 4 data docs0.982
5Claude Opus 4.8anthropic/claude-opus-4.8: adaptive-thinking-xhigh· Nejumi Leaderboard 4 data docs0.981
6Claude Opus 4.7anthropic/claude-opus-4.7: adaptive-thinking-xhigh· Nejumi Leaderboard 4 data docs0.98
7GLM 4.7zai-org/GLM-4.7: reasoning-enabled· Nejumi Leaderboard 4 data docs0.979
8Gemini 3.5 Flashgoogle/gemini-3.5-flash· Nejumi Leaderboard 4 data docs0.978
9Gemma 4 31B ITgoogle/gemma-4-31b-it· Nejumi Leaderboard 4 data docs0.976
10MiniMax M3MiniMaxAI/MiniMax-M3: reasoning-enabled· Nejumi Leaderboard 4 data docs0.976
11Claude Opus 4.6anthropic/claude-opus-4-6: extended-thinking· Nejumi Leaderboard 4 data docs0.976
12Gemini 3 Flashgoogle/gemini-3-flash-preview· Nejumi Leaderboard 4 data docs0.976
13Kimi K2.5moonshotai/Kimi-K2.5: reasoning-enabled· Nejumi Leaderboard 4 data docs0.976
14Gemini 3 Pro Previewgoogle/gemini-3-pro-preview· Nejumi Leaderboard 4 data docs0.975
15GPT 5.1openai/gpt-5.1-2025-11-13: high-effort· Nejumi Leaderboard 4 data docs0.974
16Gemma 4 26B A4Bgoogle/gemma-4-26B-A4B-it· Nejumi Leaderboard 4 data docs0.972
17DeepSeek R1deepseek-ai/DeepSeek-R1-0528: reasoning-enabled· Nejumi Leaderboard 4 data docs0.969
18GLM 5.2zai-org/GLM-5.2: reasoning-max· Nejumi Leaderboard 4 data docs0.968
19OpenAI o3openai/o3-2025-04-16: high-effort· Nejumi Leaderboard 4 data docs0.966
20Qwen3.5 27BQwen/Qwen3.5-27B: reasoning-enabled· Nejumi Leaderboard 4 data docs0.966
21Gemini 3.1 Flash Litegoogle/gemini-3.1-flash-lite-preview· Nejumi Leaderboard 4 data docs0.965
22GPT 5.1openai/gpt-5.1-2025-11-13: none-effort· Nejumi Leaderboard 4 data docs0.964
23GPT-5.6 Terraopenai/gpt-5.6-terra: max-effort· Nejumi Leaderboard 4 data docs0.964
24Qwen 3 235B A22B Thinking 2507Qwen/Qwen3-235B-A22B-Thinking-2507: reasoning-enabled· Nejumi Leaderboard 4 data docs0.964
25Grok 4.20xai/grok-4.20-0309-reasoning· Nejumi Leaderboard 4 data docs0.963
26GPT-5.6 Solopenai/gpt-5.6-sol: max-effort· Nejumi Leaderboard 4 data docs0.961
27GPT 5.2openai/gpt-5.2-2025-12-11: xhigh-effort· Nejumi Leaderboard 4 data docs0.959
28DeepSeek V4 Prodeepseek-ai/DeepSeek-V4-Pro: reasoning-max· Nejumi Leaderboard 4 data docs0.959
29GPT 4.1openai/gpt-4-1-2025-04-14· Nejumi Leaderboard 4 data docs0.958
30GLM 4.6zai-org/GLM-4.6-FP8: reasoning-enabled· Nejumi Leaderboard 4 data docs0.958
31DeepSeek Reasonerdeepseek/DeepSeek-V3.2 (Thinking Mode)· Nejumi Leaderboard 4 data docs0.958
32GLM 5zai-org/GLM-5: reasoning-enabled· Nejumi Leaderboard 4 data docs0.958
33GLM 5.2z-ai/glm-5.2: openrouter-reasoning-xhigh· Nejumi Leaderboard 4 data docs0.958
34GPT 5.4openai/gpt-5.4-2026-03-05: xhigh-effort· Nejumi Leaderboard 4 data docs0.958
35o3 Miniopenai/o3-mini-2025-01-31· Nejumi Leaderboard 4 data docs0.958
36Gemini 2.5 Progoogle/gemini-2.5-pro· Nejumi Leaderboard 4 data docs0.957
37GPT OSS 120Bopenai/gpt-oss-120b: reasoning-enabled· Nejumi Leaderboard 4 data docs0.956
38Qwen 3 235B A22BQwen/Qwen3-235B-A22B: reasoning-enabled· Nejumi Leaderboard 4 data docs0.956
39Grok 4.5xai/grok-4.5: reasoning-high· Nejumi Leaderboard 4 data docs0.956
40GPT-5.6 Lunaopenai/gpt-5.6-luna: max-effort· Nejumi Leaderboard 4 data docs0.954
41o4 Miniopenai/o4-mini-2025-04-16· Nejumi Leaderboard 4 data docs0.954
42Qwen3.5 35B A3BQwen/Qwen3.5-35B-A3B: reasoning-enabled· Nejumi Leaderboard 4 data docs0.954
43Qwen3.5 397B A17BQwen/Qwen3.5-397B-A17B: reasoning-enabled· Nejumi Leaderboard 4 data docs0.954
44Qwen3.6 27BQwen/Qwen3.6-27B: reasoning-enabled· Nejumi Leaderboard 4 data docs0.954
45Gemini 3.1 Pro Previewgoogle/gemini-3.1-pro-preview· Nejumi Leaderboard 4 data docs0.953
46GPT 5.5openai/gpt-5.5-2026-04-23: high-effort· Nejumi Leaderboard 4 data docs0.952
47DeepSeek V4 Prodeepseek/deepseek-v4-pro: thinking-max· Nejumi Leaderboard 4 data docs0.952
48Qwen3.6 35B A3BQwen/Qwen3.6-35B-A3B: reasoning-enabled· Nejumi Leaderboard 4 data docs0.951
49o1openai/o1-2024-12-17: high-effort· Nejumi Leaderboard 4 data docs0.951
50Qwen3 Max PreviewQwen/Qwen3-Max-Preview· Nejumi Leaderboard 4 data docs0.951

Showing 1–50 of 126 results.

Data credits: Nejumi Leaderboard 4, Weights & Biases Japan; public publisher-selected numeric evaluation results