Skip to content

Benchmark

VITA-Bench OTA

Vision benchmarks measure how well models understand images. Scores are percentages.

Models measured
18
Scores recorded
26
Sources
1
IQ weight
1.82

Score distribution

24.5%
Mean
26.3%
Median
9.7%
Std dev
15.8%
10th percentile
36.5%
90th percentile
4.5%
Min
4.5%40.3%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.82 for chat models, measured over 18 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Gemini 3 Pro Previewthinking high· VitaBench40.3%
2Gemini 3 Pro Previewnon-thinking low· VitaBench39.5%
3Claude Opus 4.5thinking· VitaBench36.5%
4Gemini 3 Flashthinking high· VitaBench36.5%
5Claude Opus 4.5non-thinking· VitaBench35.5%
6DeepSeek V3.2thinking· VitaBench32.3%
7Claude Sonnet 4.5non-thinking· VitaBench31.3%
8Claude Sonnet 4.5thinking· VitaBench28.3%
9GPT 5.2thinking xhigh· VitaBench28.3%
10DeepSeek V3.2non-thinking· VitaBench26.8%
11GLM 4.7thinking· VitaBench26.8%
12LongCat Flash Chatnon-thinking· VitaBench26.4%
13OpenAI o3thinking high· VitaBench26.3%
14Doubao Seed 1.8thinking high, thinking· VitaBench25.5%
15LongCat-Flash-Thinkingthinking· VitaBench25.3%
16Kimi K2 Thinkingthinking· VitaBench24%
17o4 Minithinking high· VitaBench23.5%
18GLM 4.7non-thinking· VitaBench21.3%
19Qwen3 Maxnon-thinking· VitaBench20.8%
20Kimi K2 0905non-thinking· VitaBench18.8%
21Qwen3 235B A22B Instruct 2507non-thinking· VitaBench17.3%
22Doubao Seed 1.8non-thinking· VitaBench16.5%
23Qwen 3 235B A22B Thinking 2507thinking· VitaBench15.8%
24Qwen3 32Bnon-thinking· VitaBench5.5%
25GPT 5.2non-thinking none· VitaBench4.5%
26Qwen3 32Bthinking· VitaBench4.5%