Skip to content

Benchmark

Vals MMLU Pro

Scores are percentages. Higher is better.

Models measured
121
Scores recorded
126
Sources
1
IQ weight
2.67

Score distribution

81.5%
Mean
84.2%
Median
9.3%
Std dev
69.6%
10th percentile
89.2%
90th percentile
30.3%
Min
30.3%92.4%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 2.67 for chat models, measured over 121 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
51Kimi K2.5thinking· Vals AI85.9%
52Grok 4.3· Vals AI85.8%
53Gemini 3.5 Flash Lite· Vals AI85.8%
54Nemotron 3 Ultra· Vals AI85.8%
55OpenAI o3· Vals AI85.6%
56Inkling Small· Vals AI85.6%
57Grok 4· Vals AI85.3%
58Qwen3 Max· Vals AI85%
59DeepSeek V3.2· Vals AI84.9%
60MiMo V2.5 Pro· Vals AI84.6%
61GPT-5.4 mini· Vals AI84.6%
62Qwen 3.8 27B· Vals AI84.3%
63MiniMax M3· Vals AI84.2%
64Grok 4.1 Fast· Vals AI84.2%
65Qwen3.5 Flash· Vals AI84.1%
66Gemini 2.5 Pro· Vals AI84.1%
67Claude Sonnet 4thinking· Vals AI83.9%
68gemini-2.5-flash-preview-09-2025· Vals AI83.7%
69gemini-2.5-flash-preview-09-2025thinking· Vals AI83.7%
70Qwen3 Max Preview· Vals AI83.5%
71o1· Vals AI83.5%
72DeepSeek R1· Vals AI83.2%
73MiMo V2.5· Vals AI82.9%
74GLM 4.7· Vals AI82.7%
75Claude Sonnet 3.7thinking· Vals AI82.7%
76GPT-5 mini· Vals AI82.2%
77GLM 4.6· Vals AI82.2%
78Grok 3 Mini Fast Betahigh, reasoning· Vals AI81.4%
79Qwen 3 235B A22B· Vals AI81.2%
80GLM 4.5· Vals AI81.2%
81Kimi K2 Thinking· Vals AI81.1%
82Claude Sonnet 3.7· Vals AI80.7%
83o4 Mini· Vals AI80.6%
84GPT 4.1· Vals AI80.5%
85MiniMax M2.7· Vals AI80.4%
86MiniMax M2.5· Vals AI80.1%
87Grok 3 Mini Fast Betalow, reasoning· Vals AI80%
88Grok 3· Vals AI79.9%
89Mistral Large 3· Vals AI79.8%
90Grok 4 Fast· Vals AI79.7%
91DeepSeek V3· Vals AI79.5%
92Claude Sonnet 4· Vals AI79.4%
93Llama 4 Maverick· Vals AI79.4%
94Kimi K2 Instruct· Vals AI79.4%
95GPT OSS 120B· Vals AI79.2%
96Gemini 2.5 Flash Lite Preview 09-2025thinking· Vals AI79.1%
97Claude Haiku 4.5thinking· Vals AI78.7%
98o3 Mini· Vals AI78.7%
99Gemini 2.5 Flash Lite Preview 09-2025· Vals AI78.6%
100Claude 3.5 Sonnet· Vals AI78.4%

Showing 51–100 of 126 results.