Skip to content

Benchmark

VITA-Bench In-store

Vision benchmarks measure how well models understand images. Scores are percentages.

Models measured
18
Scores recorded
26
Sources
1
IQ weight
1.93

Score distribution

42%
Mean
44.5%
Median
11%
Std dev
30%
10th percentile
55.8%
90th percentile
16.8%
Min
16.8%58.3%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.93 for chat models, measured over 18 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1LongCat-Flash-Thinkingthinking· VitaBench58.3%
2Gemini 3 Pro Previewthinking high· VitaBench56.5%
3Claude Opus 4.5thinking· VitaBench55.8%
4Gemini 3 Flashthinking high· VitaBench54.3%
5Gemini 3 Pro Previewnon-thinking low· VitaBench53.8%
6Claude Opus 4.5non-thinking· VitaBench49.8%
7Claude Sonnet 4.5thinking· VitaBench48.3%
8DeepSeek V3.2non-thinking· VitaBench48.3%
9DeepSeek V3.2thinking· VitaBench48%
10o4 Minithinking high· VitaBench46.5%
11GPT 5.2thinking xhigh· VitaBench45.8%
12OpenAI o3thinking high· VitaBench45.8%
13LongCat Flash Chatnon-thinking· VitaBench44.5%
14Claude Sonnet 4.5non-thinking· VitaBench43.8%
15GLM 4.7thinking· VitaBench42.5%
16Doubao Seed 1.8thinking high, thinking· VitaBench42%
17Doubao Seed 1.8non-thinking· VitaBench40%
18Qwen3 Maxnon-thinking· VitaBench39.8%
19GLM 4.7non-thinking· VitaBench38.8%
20Kimi K2 Thinkingthinking· VitaBench38.8%
21Qwen 3 235B A22B Thinking 2507thinking· VitaBench32%
22Qwen3 235B A22B Instruct 2507non-thinking· VitaBench31.5%
23Kimi K2 0905non-thinking· VitaBench30%
24Qwen3 32Bthinking· VitaBench22.5%
25GPT 5.2non-thinking none· VitaBench18.2%
26Qwen3 32Bnon-thinking· VitaBench16.8%