Skip to content

Benchmark

POPE

Vision benchmarks measure how well models understand images. Scores are percentages.

Models measured
36
Scores recorded
36
Sources
2
IQ weight
0.98

Score distribution

82%
Mean
85.7%
Median
14.3%
Std dev
74%
10th percentile
89%
90th percentile
2%
Min
2%90.5%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.98 for chat models, measured over 36 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1InternVL3 78B· OpenVLM Leaderboard data90.5%
2internvl3-8b· OpenVLM Leaderboard data90.4%
3GLM 4 Vision· OpenVLM Leaderboard data89.9%
4internvl3-38b· OpenVLM Leaderboard data89.2%
5Molmo 7B-D· OpenVLM Leaderboard data89%
6Gemini 1.5 Flash· OpenVLM Leaderboard data88.5%
7Gemini 1.5 Pro· OpenVLM Leaderboard data88.2%
8Llama 3.2 11B Vision Instruct· OpenVLM Leaderboard data88.1%
9Qwen2-VL-72B· OpenVLM Leaderboard data87.2%
10GLM 4 Vision Plus· OpenVLM Leaderboard data86.9%
11GPT 4.1· OpenVLM Leaderboard data86.4%
12InternVL2 26B· OpenVLM Leaderboard data86.4%
13Llama 3.2 90B Vision Instruct· OpenVLM Leaderboard data86.3%
14Phi-3.5 Vision Instruct· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim
Values from 2 sources
86.1%
15Phi 4 Multimodal Instruct· OpenVLM Leaderboard data
Values from 2 sources
85.9%
16Qwen2.5-VL 7B Instruct· OpenVLM Leaderboard data85.9%
17qwen2.5-vl-3b-instruct· OpenVLM Leaderboard data85.9%
18Hunyuan Standard Vision· OpenVLM Leaderboard data85.8%
19Qwen2.5 VL 72B Instruct· OpenVLM Leaderboard data85.7%
20GPT-4.1 mini· OpenVLM Leaderboard data85.6%
21Gemma 3 12B IT· OpenVLM Leaderboard data85.5%
22Qwen-VL Plus· OpenVLM Leaderboard data85.4%
23Gemma 3 4B· OpenVLM Leaderboard data84.6%
24InternVL2 4B· OpenVLM Leaderboard data84.6%
25Gemma 3 27B IT· OpenVLM Leaderboard data84.2%
26Pixtral 12B· OpenVLM Leaderboard data84.2%
27Yi Vision· OpenVLM Leaderboard data84.1%
28GPT-4.1 nano· OpenVLM Leaderboard data83.5%
29MiniCPM-V 2.6· OpenVLM Leaderboard data83.2%
30Grok 2 Vision 1212· OpenVLM Leaderboard data83.1%
31Claude 3 Haiku· OpenVLM Leaderboard data74.4%
32Claude 3 Opus· OpenVLM Leaderboard data74%
33Qwen-VL Max· OpenVLM Leaderboard data71.9%
34Claude 3 Sonnet· OpenVLM Leaderboard data68.2%
35kosmos-2· OpenVLM Leaderboard data66.3%
36Moonshot V1 8K· OpenVLM Leaderboard data2%

Data credits: OpenVLM Leaderboard (OpenCompass / VLMEvalKit), Apache-2.0 (licence) · Data by LLM Stats