Skip to content

Benchmark

MMMU

Vision benchmarks measure how well models understand images. Scores are percentages. Higher is better.

Models measured
76
Scores recorded
84
Sources
4
IQ weight
1.31

Score distribution

66.7%
Mean
69.4%
Median
14.3%
Std dev
50.7%
10th percentile
82.9%
90th percentile
0%
Min
0%86%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.31 for chat models, measured over 71 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
51Grok 2 mini· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim63.2%
52Mistral Small 3.2 24B Instruct2506· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim62.5%
53Gemini 1.5 Flash· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim62.3%
54GPT-4 Turbo· WildVision Arena data62.3%
55gemini-1.5-pro-latest· WildVision Arena data62.2%
56Nova Pro· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim61.7%
57Claude 3 Opus· WildVision Arena data59.4%
58GPT-4o mini· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim
Values from 2 sources
59.4%
59Mistral Small 3.1 24b Instruct 2503· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim59.3%
60Qwen2.5-Omni 7B· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim59.2%
61Qwen2.5-VL 7B Instruct· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim58.6%
62GPT-4-vision-preview· WildVision Arena data56.8%
63LLaVA-OneVision Qwen2 72B OV· WildVision Arena data56.8%
64Reka Core· WildVision Arena data56.3%
65Nova Lite· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim56.2%
66Gemini 1.5 Flashlatest· WildVision Arena data56.1%
67GPT-4.1 nano· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim55.4%
68Phi 4 Multimodal Instruct· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim55.1%
69Qwen2 VL 7b Instruct· WildVision Arena data54.1%
70Gemini 1.5 Flash-8B· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim53.7%
71Reka Flash· WildVision Arena data53.3%
72Claude 3 Sonnet· WildVision Arena data53.1%
73Pixtral 12B2409· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim
Values from 3 sources
52.5%
74Qwen-VL Max· WildVision Arena data51.4%
75DeepSeek VL2· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim51.1%
76LLaVA 1.6 34B· WildVision Arena data
Values from 2 sources
51.1%
77Llama 3.2 11B Instruct· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim50.7%
78Claude 3 Haiku· WildVision Arena data50.2%
79Gemini Pro Vision· WildVision Arena data47.9%
80gemini-1.0-pro· LLM Stats data47.9%
81Qwen-VL Plus· WildVision Arena data45.2%
82Phi-3.5 Vision Instruct· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim43%
83Phi-3 Vision 128K Instruct· WildVision Arena data40.4%
84GPT-3.5 Turbo· LLM Stats data0%

Showing 51–84 of 84 results.

Data credits: WildVision Arena, MIT (licence) · BenchBoard by SYuan03 (github.com/SYuan03/benchboard). MIT covers its code and data organization; no results licence is inferred. Numeric relays retain the original stating-source links. · Data by LLM Stats