Skip to content

Benchmark

Nejumi 4: AVG_jaster_2shots

Higher scores are better.

Models measured
114
Scores recorded
126
Sources
1
IQ weight
1.97

Score distribution

0.772
Mean
0.807
Median
0.105
Std dev
0.625
10th percentile
0.872
90th percentile
0.435
Min
0.4350.886
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.97 for chat models, measured over 114 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
101Qwen3 4BQwen/Qwen3-4B: reasoning-enabled· Nejumi Leaderboard 4 data docs0.701
102GPT-4o miniopenai/gpt-4o-mini-2024-07-18· Nejumi Leaderboard 4 data docs0.699
103Mistral Smallmistral/mistral-small-2503· Nejumi Leaderboard 4 data docs0.688
104Gemma 4 E4B-itgoogle/gemma-4-E4B-it· Nejumi Leaderboard 4 data docs0.681
105Claude Opus 4.5anthropic/claude-opus-4.5-20251125: no-thinking· Nejumi Leaderboard 4 data docs0.662
106Claude Sonnet 4anthropic/claude-sonnet-4-20250514: no-thinking· Nejumi Leaderboard 4 data docs0.657
107Gemma 3 12B ITgoogle/gemma-3-12b-it· Nejumi Leaderboard 4 data docs0.653
108Ministral 3 14Bmistralai/Ministral-3-14B-Reasoning-2512· Nejumi Leaderboard 4 data docs0.65
109ERNIE 4.5 21B A3Bbaidu/ERNIE-4.5-21B-A3B-Thinking· Nejumi Leaderboard 4 data docs0.647
110Claude Sonnet 3.7anthropic/claude-3.7-sonnet-20250219: no-thinking· Nejumi Leaderboard 4 data docs0.64
111GPT-4.1 nanoopenai/gpt-4-1-nano-2025-04-14· Nejumi Leaderboard 4 data docs0.633
112Claude 3.5 Sonnetanthropic/claude-3.5-sonnet-20241022· Nejumi Leaderboard 4 data docs0.629
113Ministral 3 8Bmistralai/Ministral-3-8B-Reasoning-2512· Nejumi Leaderboard 4 data docs0.625
114Gemma 4 E2B-itgoogle/gemma-4-E2B-it· Nejumi Leaderboard 4 data docs0.624
115Qwen3 1.7BQwen/Qwen3-1.7B: reasoning-enabled· Nejumi Leaderboard 4 data docs0.607
116Mistral Largemistral/mistral-large-2411· Nejumi Leaderboard 4 data docs0.581
117Gemini 2.5 Flash-Litegoogle/gemini-2.5-flash-lite· Nejumi Leaderboard 4 data docs0.562
118Claude Haiku 3.5anthropic/claude-3.5-haiku-20241022· Nejumi Leaderboard 4 data docs0.557
119Nova Pro 1.0us.amazon.nova-pro-v1:0· Nejumi Leaderboard 4 data docs0.55
120Nova Lite 1.0us.amazon.nova-lite-v1:0· Nejumi Leaderboard 4 data docs0.549
121Gemma 3 4B ITgoogle/gemma-3-4b-it· Nejumi Leaderboard 4 data docs0.544
122Ministral 3 3Bmistralai/Ministral-3-3B-Reasoning-2512· Nejumi Leaderboard 4 data docs0.487
123Qwen3 0.6BQwen/Qwen3-0.6B: reasoning-enabled· Nejumi Leaderboard 4 data docs0.484
124Nova Micro 1.0us.amazon.nova-micro-v1:0· Nejumi Leaderboard 4 data docs0.474
125Llama 3.2 3Bmeta-llama/Llama-3.2-3B-Instruct· Nejumi Leaderboard 4 data docs0.457
126QwQ 32BQwen/QwQ-32B· Nejumi Leaderboard 4 data docs0.435

Showing 101–126 of 126 results.

Data credits: Nejumi Leaderboard 4, Weights & Biases Japan; public publisher-selected numeric evaluation results