Skip to content

Benchmark

Nejumi 4: AVG_mtbench

People compare model answers and choose which they prefer. Higher scores are better.

Models measured
114
Scores recorded
126
Sources
1
IQ weight
2.03

Score distribution

0.908
Mean
0.942
Median
0.103
Std dev
0.826
10th percentile
0.976
90th percentile
0.396
Min
0.3960.988
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 2.03 for chat models, measured over 114 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
101GPT-4.1 nanoopenai/gpt-4-1-nano-2025-04-14· Nejumi Leaderboard 4 data docs0.879
102Claude Haiku 3.5anthropic/claude-3.5-haiku-20241022· Nejumi Leaderboard 4 data docs0.873
103Llama 4 Maverickmeta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8· Nejumi Leaderboard 4 data docs0.87
104Qwen3 4BQwen/Qwen3-4B: reasoning-enabled· Nejumi Leaderboard 4 data docs0.868
105GPT-4o miniopenai/gpt-4o-mini-2024-07-18· Nejumi Leaderboard 4 data docs0.859
106Nova Pro 1.0us.amazon.nova-pro-v1:0· Nejumi Leaderboard 4 data docs0.857
107Mistral Largemistral/mistral-large-2411· Nejumi Leaderboard 4 data docs0.849
108Ministral 3 14Bmistralai/Ministral-3-14B-Reasoning-2512· Nejumi Leaderboard 4 data docs0.848
109Llama 3.1 405B Instructmeta-llama/Llama-3.1-405B-Instruct-FP8· Nejumi Leaderboard 4 data docs0.836
110Qwen3-4B-Thinking-2507Qwen/Qwen3-4B-Thinking-2507· Nejumi Leaderboard 4 data docs0.834
111Qwen3-4B-Thinking-2507Qwen/Qwen3-4B-Thinking-2507: reasoning-enabled· Nejumi Leaderboard 4 data docs0.834
112Ministral 3 8Bmistralai/Ministral-3-8B-Reasoning-2512· Nejumi Leaderboard 4 data docs0.83
113Gemma 3 12B ITgoogle/gemma-3-12b-it· Nejumi Leaderboard 4 data docs0.826
114Llama 4 Scoutmeta-llama/Llama-4-Scout-17B-16E-Instruct· Nejumi Leaderboard 4 data docs0.824
115Llama 3.3 70B Instructmeta-llama/Llama-3.3-70B-Instruct· Nejumi Leaderboard 4 data docs0.786
116Nova Micro 1.0us.amazon.nova-micro-v1:0· Nejumi Leaderboard 4 data docs0.784
117ERNIE 4.5 21B A3Bbaidu/ERNIE-4.5-21B-A3B-Thinking· Nejumi Leaderboard 4 data docs0.776
118Gemini 2.5 Flashgoogle/gemini-2.5-flash· Nejumi Leaderboard 4 data docs0.774
119Nova Lite 1.0us.amazon.nova-lite-v1:0· Nejumi Leaderboard 4 data docs0.766
120Gemma 3 4B ITgoogle/gemma-3-4b-it· Nejumi Leaderboard 4 data docs0.753
121Qwen3 1.7BQwen/Qwen3-1.7B: reasoning-enabled· Nejumi Leaderboard 4 data docs0.672
122Ministral 3 3Bmistralai/Ministral-3-3B-Reasoning-2512· Nejumi Leaderboard 4 data docs0.647
123Kimi K2 Thinkingmoonshotai/kimi-k2-thinking· Nejumi Leaderboard 4 data docs0.63
124Mistral Smallmistral/mistral-small-2503· Nejumi Leaderboard 4 data docs0.416
125Qwen3 0.6BQwen/Qwen3-0.6B: reasoning-enabled· Nejumi Leaderboard 4 data docs0.404
126Llama 3.2 3Bmeta-llama/Llama-3.2-3B-Instruct· Nejumi Leaderboard 4 data docs0.396

Showing 101–126 of 126 results.

Data credits: Nejumi Leaderboard 4, Weights & Biases Japan; public publisher-selected numeric evaluation results