Skip to content

Benchmark

VideoMME w/o sub.

Vision benchmarks measure how well models understand images. Scores are percentages.

Models measured
8
Scores recorded
9
Sources
1
IQ weight
2.08

Score distribution

78.4%
Mean
79.2%
Median
5.7%
Std dev
73.3%
10th percentile
82.8%
90th percentile
65.1%
Min
65.1%83.9%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 2.08 for chat models, measured over 8 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Qwen3.5 122B-A10B· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim83.9%
2Qwen3.5 27B· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim82.8%
3Qwen3.5 35B A3B· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim82.5%
4Qwen3.6 35B A3B· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim82.5%
5Qwen3 VL 235B· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it.79.2%
6Qwen3 VL 235Bthinking· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it.79%
7Qwen3 VL 32B Thinking· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it.77.3%
8Qwen2.5 VL 72B Instruct· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim73.3%
9Qwen2.5-VL 7B Instruct· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim65.1%

Data credits: Data by LLM Stats