Skip to content

Benchmark

SEED-Bench 2 Plus

Vision benchmarks measure how well models understand images. Scores are percentages.

Models measured
34
Scores recorded
34
Sources
1
IQ weight
1.11

Score distribution

65.1%
Mean
67.6%
Median
10.4%
Std dev
60.7%
10th percentile
71.9%
90th percentile
25.3%
Min
25.3%73.8%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.11 for chat models, measured over 34 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Qwen2.5 VL 72B Instruct· OpenVLM Leaderboard data73.8%
2GPT 4.1· OpenVLM Leaderboard data73.1%
3Qwen2-VL-72B· OpenVLM Leaderboard data72.3%
4GPT-4.1 mini· OpenVLM Leaderboard data71.9%
5GLM 4 Vision Plus· OpenVLM Leaderboard data71.8%
6internvl3-38b· OpenVLM Leaderboard data71.8%
7InternVL3 78B· OpenVLM Leaderboard data71.7%
8Gemini 1.5 Pro· OpenVLM Leaderboard data70.8%
9Qwen2.5-VL 7B Instruct· OpenVLM Leaderboard data70.7%
10internvl3-8b· OpenVLM Leaderboard data69.4%
11Qwen-VL Plus· OpenVLM Leaderboard data69.2%
12Hunyuan Standard Vision· OpenVLM Leaderboard data69%
13Gemini 1.5 Flash· OpenVLM Leaderboard data68.6%
14Phi 4 Multimodal Instruct· OpenVLM Leaderboard data68.5%
15qwen2.5-vl-3b-instruct· OpenVLM Leaderboard data68.5%
16Llama 3.2 90B Vision Instruct· OpenVLM Leaderboard data68.2%
17InternVL2 26B· OpenVLM Leaderboard data67.6%
18Molmo 7B-D· OpenVLM Leaderboard data67.6%
19Pixtral 12B· OpenVLM Leaderboard data67.4%
20Gemma 3 27B IT· OpenVLM Leaderboard data67%
21GPT-4.1 nano· OpenVLM Leaderboard data67%
22Llama 3.2 11B Vision Instruct· OpenVLM Leaderboard data65.8%
23MiniCPM-V 2.6· OpenVLM Leaderboard data65.7%
24Moonshot V1 8K· OpenVLM Leaderboard data65.3%
25Gemma 3 12B IT· OpenVLM Leaderboard data65.2%
26InternVL2 4B· OpenVLM Leaderboard data63.7%
27Phi-3.5 Vision Instruct· OpenVLM Leaderboard data62.2%
28Yi Vision· OpenVLM Leaderboard data62.1%
29Claude 3 Haiku· OpenVLM Leaderboard data61.7%
30Qwen-VL Max· OpenVLM Leaderboard data61.6%
31Gemma 3 4B· OpenVLM Leaderboard data60.7%
32Grok 2 Vision 1212· OpenVLM Leaderboard data60.4%
33Claude 3 Sonnet· OpenVLM Leaderboard data27.1%
34kosmos-2· OpenVLM Leaderboard data25.3%

Data credits: OpenVLM Leaderboard (OpenCompass / VLMEvalKit), Apache-2.0 (licence)