Skip to content

Benchmark

MMMU (val)

Vision benchmarks measure how well models understand images. Scores are percentages.

Models measured
58
Scores recorded
60
Sources
2
IQ weight
0.92

Score distribution

60.9%
Mean
60.6%
Median
12.4%
Std dev
47.4%
10th percentile
76%
90th percentile
23.7%
Min
23.7%81.8%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.92 for chat models, measured over 58 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1GPT 5· OpenVLM Leaderboard data81.8%
2Qwen3 VL 235Bthinking· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it.80.6%
3GPT-5 mini· OpenVLM Leaderboard data78.7%
4Qwen3 VL 235B· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it.78.7%
5Qwen3 VL 32B Thinking· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it.78.1%
6Claude Sonnet 4.5· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim77.8%
7Qwen3 VL 32B Instruct· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it.76%
8qwen3-vl-30bthinking· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it.76%
9Gemini 2.5 Pro· OpenVLM Leaderboard data74.7%
10qwen3-vl-30b· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it.74.2%
11Qwen: Qwen3 VL 8B Thinking· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it.74.1%
12GPT 4.1· OpenVLM Leaderboard data74%
13ChatGPT-4o Latest· OpenVLM Leaderboard data72.9%
14GPT-5 nano· OpenVLM Leaderboard data72.6%
15InternVL3 78B· OpenVLM Leaderboard data72.2%
16GPT-4.5 Preview· OpenVLM Leaderboard data72.1%
17Hunyuan Standard Vision· OpenVLM Leaderboard data70.7%
18SenseNova-V6-Pro· OpenVLM Leaderboard data70.4%
19Gemini 2.0 Flash· OpenVLM Leaderboard data69.9%
20internvl3-38b· OpenVLM Leaderboard data69.7%
21Qwen3 VL 8B Instruct· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it.69.6%
22gemini-1.5-pro-002· OpenVLM Leaderboard data68.6%
23Qwen2.5 VL 72B Instruct· OpenVLM Leaderboard data68.2%
24Grok 2 Vision 1212· OpenVLM Leaderboard data67.1%
25Gemma 3 27B IT· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim
Values from 2 sources
64.9%
26Qwen2-VL-72B· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim
Values from 2 sources
64.5%
27internvl3-8b· OpenVLM Leaderboard data62.2%
28GLM 4 Vision Plus· OpenVLM Leaderboard data61.7%
29gemini-1.5-flash-002· OpenVLM Leaderboard data60.7%
30Gemini 1.5 Pro· OpenVLM Leaderboard data60.6%
31Llama 3.2 90B Vision Instruct· OpenVLM Leaderboard data60.3%
32Yi Vision· OpenVLM Leaderboard data60.3%
33Gemma 3 12B IT· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim
Values from 2 sources
59.6%
34Gemini 1.5 Flash· OpenVLM Leaderboard data58.2%
35Qwen2.5-VL 7B Instruct· OpenVLM Leaderboard data58%
36Moonshot V1 8K· OpenVLM Leaderboard data57.7%
37GPT-4.1 nano· OpenVLM Leaderboard data57.6%
38Phi 4 Multimodal Instruct· OpenVLM Leaderboard data56%
39GPT-4.1 mini· OpenVLM Leaderboard data55%
40Claude 3 Opus· OpenVLM Leaderboard data54.9%
41DeepSeek VL2· OpenVLM Leaderboard data54%
42Molmo 72B· OpenVLM Leaderboard data52.8%
43Qwen-VL Max· OpenVLM Leaderboard data52%
44qwen2.5-vl-3b-instruct· OpenVLM Leaderboard data51.2%
45Pixtral 12B· OpenVLM Leaderboard data51.1%
46InternVL2 26B· OpenVLM Leaderboard data50.7%
47MiniCPM-V 2.6· OpenVLM Leaderboard data49.8%
48Claude 3 Haiku· OpenVLM Leaderboard data49.7%
49Molmo 7B-D· OpenVLM Leaderboard data49.1%
50gemini-1.0-pro· OpenVLM Leaderboard data49%

Showing 1–50 of 60 results.

Data credits: OpenVLM Leaderboard (OpenCompass / VLMEvalKit), Apache-2.0 (licence) · Data by LLM Stats