Skip to content

Benchmark

Nejumi 4: AVG_jaster_0shot

Higher scores are better.

Models measured
114
Scores recorded
126
Sources
1
IQ weight
2.02

Score distribution

0.754
Mean
0.796
Median
0.122
Std dev
0.547
10th percentile
0.861
90th percentile
0.317
Min
0.3170.88
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 2.02 for chat models, measured over 114 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
101Gemma 3 27B ITgoogle/gemma-3-27b-it· Nejumi Leaderboard 4 data docs0.671
102Llama 3.3 70B Instructmeta-llama/Llama-3.3-70B-Instruct· Nejumi Leaderboard 4 data docs0.662
103Mistral Smallmistral/mistral-small-2503· Nejumi Leaderboard 4 data docs0.642
104Gemma 3 12B ITgoogle/gemma-3-12b-it· Nejumi Leaderboard 4 data docs0.634
105Llama 4 Scoutmeta-llama/Llama-4-Scout-17B-16E-Instruct· Nejumi Leaderboard 4 data docs0.632
106Claude Sonnet 3.7anthropic/claude-3.7-sonnet-20250219: no-thinking· Nejumi Leaderboard 4 data docs0.619
107GPT-4.1 nanoopenai/gpt-4-1-nano-2025-04-14· Nejumi Leaderboard 4 data docs0.619
108Claude Opus 4.5anthropic/claude-opus-4.5-20251125: no-thinking· Nejumi Leaderboard 4 data docs0.616
109Qwen3 1.7BQwen/Qwen3-1.7B: reasoning-enabled· Nejumi Leaderboard 4 data docs0.615
110Llama 4 Maverickmeta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8· Nejumi Leaderboard 4 data docs0.61
111Claude Sonnet 4anthropic/claude-sonnet-4-20250514: no-thinking· Nejumi Leaderboard 4 data docs0.608
112Claude 3.5 Sonnetanthropic/claude-3.5-sonnet-20241022· Nejumi Leaderboard 4 data docs0.607
113Mistral Largemistral/mistral-large-2411· Nejumi Leaderboard 4 data docs0.547
114Gemma 3 4B ITgoogle/gemma-3-4b-it· Nejumi Leaderboard 4 data docs0.544
115Ministral 3 14Bmistralai/Ministral-3-14B-Reasoning-2512· Nejumi Leaderboard 4 data docs0.538
116Gemini 2.5 Flash-Litegoogle/gemini-2.5-flash-lite· Nejumi Leaderboard 4 data docs0.535
117Ministral 3 8Bmistralai/Ministral-3-8B-Reasoning-2512· Nejumi Leaderboard 4 data docs0.529
118ERNIE 4.5 21B A3Bbaidu/ERNIE-4.5-21B-A3B-Thinking· Nejumi Leaderboard 4 data docs0.522
119Claude Haiku 3.5anthropic/claude-3.5-haiku-20241022· Nejumi Leaderboard 4 data docs0.509
120Qwen3 0.6BQwen/Qwen3-0.6B: reasoning-enabled· Nejumi Leaderboard 4 data docs0.486
121Nova Pro 1.0us.amazon.nova-pro-v1:0· Nejumi Leaderboard 4 data docs0.476
122Nova Micro 1.0us.amazon.nova-micro-v1:0· Nejumi Leaderboard 4 data docs0.461
123Nova Lite 1.0us.amazon.nova-lite-v1:0· Nejumi Leaderboard 4 data docs0.439
124QwQ 32BQwen/QwQ-32B· Nejumi Leaderboard 4 data docs0.426
125Ministral 3 3Bmistralai/Ministral-3-3B-Reasoning-2512· Nejumi Leaderboard 4 data docs0.365
126Llama 3.2 3Bmeta-llama/Llama-3.2-3B-Instruct· Nejumi Leaderboard 4 data docs0.317

Showing 101–126 of 126 results.

Data credits: Nejumi Leaderboard 4, Weights & Biases Japan; public publisher-selected numeric evaluation results