Skip to content

Benchmark

VITA-Bench Delivery

Vision benchmarks measure how well models understand images. Scores are percentages.

Models measured
18
Scores recorded
26
Sources
1
IQ weight
1.78

Score distribution

44.6%
Mean
45.3%
Median
11.4%
Std dev
32.5%
10th percentile
60.5%
90th percentile
18%
Min
18%62%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.78 for chat models, measured over 18 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Gemini 3 Pro Previewthinking high· VitaBench62%
2Gemini 3 Pro Previewnon-thinking low· VitaBench61%
3Claude Opus 4.5thinking· VitaBench60.5%
4Gemini 3 Flashthinking high· VitaBench58.3%
5Claude Opus 4.5non-thinking· VitaBench56.5%
6DeepSeek V3.2thinking· VitaBench53.8%
7GPT 5.2thinking xhigh· VitaBench52.5%
8Claude Sonnet 4.5thinking· VitaBench52%
9LongCat-Flash-Thinkingthinking· VitaBench52%
10OpenAI o3thinking high· VitaBench51.3%
11LongCat Flash Chatnon-thinking· VitaBench46.7%
12Doubao Seed 1.8thinking high, thinking· VitaBench46%
13DeepSeek V3.2non-thinking· VitaBench45.3%
14Qwen 3 235B A22B Thinking 2507thinking· VitaBench45%
15GLM 4.7thinking· VitaBench44.5%
16o4 Minithinking high· VitaBench44.5%
17Claude Sonnet 4.5non-thinking· VitaBench42.5%
18Doubao Seed 1.8non-thinking· VitaBench39.8%
19GLM 4.7non-thinking· VitaBench37.8%
20Kimi K2 Thinkingthinking· VitaBench36.3%
21Qwen3 Maxnon-thinking· VitaBench35.5%
22Kimi K2 0905non-thinking· VitaBench32.5%
23Qwen3 235B A22B Instruct 2507non-thinking· VitaBench32.5%
24Qwen3 32Bthinking· VitaBench27%
25GPT 5.2non-thinking none· VitaBench24.8%
26Qwen3 32Bnon-thinking· VitaBench18%