Skip to content

Benchmark

SEED-Bench IMG

Vision benchmarks measure how well models understand images. Scores are percentages.

Models measured
38
Scores recorded
38
Sources
1
IQ weight
1.08

Score distribution

72.3%
Mean
74%
Median
8.8%
Std dev
65.5%
10th percentile
77.9%
90th percentile
25.1%
Min
25.1%79.5%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.08 for chat models, measured over 38 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Qwen2.5 VL 72B Instruct· OpenVLM Leaderboard data79.5%
2InternVL3 78B· OpenVLM Leaderboard data78.7%
3internvl3-38b· OpenVLM Leaderboard data78.3%
4GPT 4.1· OpenVLM Leaderboard data78%
5Qwen2-VL-72B· OpenVLM Leaderboard data77.9%
6GPT-4.1 mini· OpenVLM Leaderboard data77.6%
7internvl3-8b· OpenVLM Leaderboard data77.1%
8DeepSeek VL2· OpenVLM Leaderboard data77%
9Qwen2.5-VL 7B Instruct· OpenVLM Leaderboard data77%
10Llama 3.2 90B Vision Instruct· OpenVLM Leaderboard data76.8%
11GLM 4 Vision Plus· OpenVLM Leaderboard data76.5%
12Grok 2 Vision 1212· OpenVLM Leaderboard data76.3%
13InternVL2 26B· OpenVLM Leaderboard data76.3%
14Gemini 1.5 Pro· OpenVLM Leaderboard data76%
15Moonshot V1 8K· OpenVLM Leaderboard data75.2%
16GLM 4 Vision· OpenVLM Leaderboard data75.1%
17Gemini 1.5 Flash· OpenVLM Leaderboard data75%
18Molmo 7B-D· OpenVLM Leaderboard data74.6%
19MiniCPM-V 2.6· OpenVLM Leaderboard data74%
20qwen2.5-vl-3b-instruct· OpenVLM Leaderboard data74%
21Yi Vision· OpenVLM Leaderboard data73.6%
22Hunyuan Standard Vision· OpenVLM Leaderboard data73.2%
23InternVL2 4B· OpenVLM Leaderboard data73.2%
24Phi 4 Multimodal Instruct· OpenVLM Leaderboard data73.2%
25Gemma 3 27B IT· OpenVLM Leaderboard data73.1%
26Llama 3.2 11B Vision Instruct· OpenVLM Leaderboard data72.7%
27Qwen-VL Max· OpenVLM Leaderboard data72.7%
28GPT-4.1 nano· OpenVLM Leaderboard data71.8%
29Pixtral 12B· OpenVLM Leaderboard data71.5%
30Gemma 3 12B IT· OpenVLM Leaderboard data71%
31gemini-1.0-pro· OpenVLM Leaderboard data70.7%
32Phi-3.5 Vision Instruct· OpenVLM Leaderboard data69.7%
33Qwen-VL Plus· OpenVLM Leaderboard data65.7%
34Gemma 3 4B· OpenVLM Leaderboard data65.5%
35Claude 3 Sonnet· OpenVLM Leaderboard data65%
36Claude 3 Opus· OpenVLM Leaderboard data64%
37Claude 3 Haiku· OpenVLM Leaderboard data63.3%
38kosmos-2· OpenVLM Leaderboard data25.1%

Data credits: OpenVLM Leaderboard (OpenCompass / VLMEvalKit), Apache-2.0 (licence)