Benchmark
Nejumi 4: AVG_mtbench
People compare model answers and choose which they prefer. Higher scores are better.
- Models measured
- 114
- Scores recorded
- 126
- Sources
- 1
- IQ weight
- 2.03
Score distribution
- 0.908
- Mean
- 0.942
- Median
- 0.103
- Std dev
- 0.826
- 10th percentile
- 0.976
- 90th percentile
- 0.396
- Min
This benchmark has a weight of 2.03 for chat models, measured over 114 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.
Leaderboard
| # | Model | Score |
|---|---|---|
| 101 | GPT-4.1 nanoopenai/gpt-4-1-nano-2025-04-14· Nejumi Leaderboard 4 data docs | 0.879 |
| 102 | Claude Haiku 3.5anthropic/claude-3.5-haiku-20241022· Nejumi Leaderboard 4 data docs | 0.873 |
| 103 | Llama 4 Maverickmeta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8· Nejumi Leaderboard 4 data docs | 0.87 |
| 104 | Qwen3 4BQwen/Qwen3-4B: reasoning-enabled· Nejumi Leaderboard 4 data docs | 0.868 |
| 105 | GPT-4o miniopenai/gpt-4o-mini-2024-07-18· Nejumi Leaderboard 4 data docs | 0.859 |
| 106 | Nova Pro 1.0us.amazon.nova-pro-v1:0· Nejumi Leaderboard 4 data docs | 0.857 |
| 107 | Mistral Largemistral/mistral-large-2411· Nejumi Leaderboard 4 data docs | 0.849 |
| 108 | Ministral 3 14Bmistralai/Ministral-3-14B-Reasoning-2512· Nejumi Leaderboard 4 data docs | 0.848 |
| 109 | Llama 3.1 405B Instructmeta-llama/Llama-3.1-405B-Instruct-FP8· Nejumi Leaderboard 4 data docs | 0.836 |
| 110 | Qwen3-4B-Thinking-2507Qwen/Qwen3-4B-Thinking-2507· Nejumi Leaderboard 4 data docs | 0.834 |
| 111 | Qwen3-4B-Thinking-2507Qwen/Qwen3-4B-Thinking-2507: reasoning-enabled· Nejumi Leaderboard 4 data docs | 0.834 |
| 112 | Ministral 3 8Bmistralai/Ministral-3-8B-Reasoning-2512· Nejumi Leaderboard 4 data docs | 0.83 |
| 113 | Gemma 3 12B ITgoogle/gemma-3-12b-it· Nejumi Leaderboard 4 data docs | 0.826 |
| 114 | Llama 4 Scoutmeta-llama/Llama-4-Scout-17B-16E-Instruct· Nejumi Leaderboard 4 data docs | 0.824 |
| 115 | Llama 3.3 70B Instructmeta-llama/Llama-3.3-70B-Instruct· Nejumi Leaderboard 4 data docs | 0.786 |
| 116 | Nova Micro 1.0us.amazon.nova-micro-v1:0· Nejumi Leaderboard 4 data docs | 0.784 |
| 117 | ERNIE 4.5 21B A3Bbaidu/ERNIE-4.5-21B-A3B-Thinking· Nejumi Leaderboard 4 data docs | 0.776 |
| 118 | Gemini 2.5 Flashgoogle/gemini-2.5-flash· Nejumi Leaderboard 4 data docs | 0.774 |
| 119 | Nova Lite 1.0us.amazon.nova-lite-v1:0· Nejumi Leaderboard 4 data docs | 0.766 |
| 120 | Gemma 3 4B ITgoogle/gemma-3-4b-it· Nejumi Leaderboard 4 data docs | 0.753 |
| 121 | Qwen3 1.7BQwen/Qwen3-1.7B: reasoning-enabled· Nejumi Leaderboard 4 data docs | 0.672 |
| 122 | Ministral 3 3Bmistralai/Ministral-3-3B-Reasoning-2512· Nejumi Leaderboard 4 data docs | 0.647 |
| 123 | Kimi K2 Thinkingmoonshotai/kimi-k2-thinking· Nejumi Leaderboard 4 data docs | 0.63 |
| 124 | Mistral Smallmistral/mistral-small-2503· Nejumi Leaderboard 4 data docs | 0.416 |
| 125 | Qwen3 0.6BQwen/Qwen3-0.6B: reasoning-enabled· Nejumi Leaderboard 4 data docs | 0.404 |
| 126 | Llama 3.2 3Bmeta-llama/Llama-3.2-3B-Instruct· Nejumi Leaderboard 4 data docs | 0.396 |
Showing 101–126 of 126 results.
Data credits: Nejumi Leaderboard 4, Weights & Biases Japan; public publisher-selected numeric evaluation results