Skip to content

Benchmark

Video-MME-v2 - Capability Consistency

Vision benchmarks measure how well models understand images. Scores are percentages.

Models measured
20
Scores recorded
78
Sources
1
IQ weight
1.54

Score distribution

23%
Mean
20.9%
Median
9.1%
Std dev
13.3%
10th percentile
37.1%
90th percentile
8.5%
Min
8.5%50.8%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.54 for chat models, measured over 20 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Gemini 3 Pro Previeww. sub @ 1fps· Video-MME-v250.8%
2Gemini 3 Flashw. sub @ 1fps· Video-MME-v244.9%
3Doubao Seed 2.0 Prow. sub @ 1fps· Video-MME-v243.7%
4Kimi K2.5w. sub @ 512· Video-MME-v240.6%
5Gemini 3 Pro Previewwo sub @ 1fps· Video-MME-v239.7%
6MiMo V2 Omniw. sub @ 1fps· Video-MME-v239.7%
7Qwen3.5 397B A17Bw. sub @ 512, think· Video-MME-v239%
8GPT 5w. sub @ 50· Video-MME-v237.4%
9Doubao Seed 2.0 Prowo sub @ 1fps· Video-MME-v237.1%
10Kimi K2.5w. sub @ 64· Video-MME-v236%
11Gemini 3 Flashwo sub @ 1fps· Video-MME-v235.6%
12Kimi K2.5wo sub @ 512· Video-MME-v234%
13MiMo V2 Omniwo sub @ 1fps· Video-MME-v232.4%
14Qwen3.5 397B A17Bwo sub @ 512, think· Video-MME-v232.2%
15Qwen3.5 27Bw. sub @ 512, think· Video-MME-v231.5%
16Qwen3.5 397B A17Bw. sub @ 64, think· Video-MME-v229.5%
17Kimi K2.5wo sub @ 64· Video-MME-v228.7%
18Qwen3.5 27Bw. sub @ 64, think· Video-MME-v228.5%
19Qwen3 VL 235Bw. sub @ 512, think· Video-MME-v228.3%
20GPT 5wo sub @ 50· Video-MME-v228.1%
21Qwen3.5 Omni Plusw. sub @ 512· Video-MME-v227.6%
22Gemma 4 31B ITw. sub @ 60· Video-MME-v226.7%
23Qwen3.5 122B-A10Bw. sub @ 64, think· Video-MME-v226.4%
24Qwen3 VL 235Bw. sub @ 64, think· Video-MME-v226%
25Qwen3.5 9Bw. sub @ 512, think· Video-MME-v225.8%
26Qwen3.5 Omni Plusw. sub @ 64· Video-MME-v225.4%
27Qwen3 VL 235Bw. sub @ 64, instruct· Video-MME-v225%
28Qwen3.5 35B A3Bw. sub @ 64, think· Video-MME-v224.9%
29Qwen3.5 122B-A10Bw. sub @ 512, think· Video-MME-v224.3%
30Qwen3.5 397B A17Bw. sub @ 64, instruct· Video-MME-v223.9%
31Qwen3.5 9Bw. sub @ 64, think· Video-MME-v223.6%
32Qwen3.5 27Bw. sub @ 64, instruct· Video-MME-v223.4%
33Qwen3.5 35B A3Bw. sub @ 512, think· Video-MME-v223.4%
34internvl3-5-241b-a28bw. sub @ 64, instruct· Video-MME-v223.4%
35qwen3-vl-30bw. sub @ 512, think· Video-MME-v221.7%
36Qwen3.5 Omni Pluswo sub @ 512· Video-MME-v221.5%
37Qwen3.5 9Bwo sub @ 512, think· Video-MME-v221%
38Qwen2.5 VL 72B Instructw. sub @ 64, instruct· Video-MME-v220.9%
39Qwen3.5 397B A17Bwo sub @ 64, think· Video-MME-v220.9%
40Qwen3.5 27Bwo sub @ 512, think· Video-MME-v220.6%
41GLM 4.5Vw. sub @ 64, think· Video-MME-v220.5%
42Qwen3 VL 235Bwo sub @ 512, think· Video-MME-v220.4%
43Qwen3.5 4Bw. sub @ 64, think· Video-MME-v220.4%
44qwen3-vl-30bw. sub @ 64, think· Video-MME-v220.2%
45Qwen: Qwen3 VL 8B Thinkingw. sub @ 512, think· Video-MME-v220.1%
46Qwen3.5 122B-A10Bw. sub @ 64, instruct· Video-MME-v220.1%
47Gemma 4 31B ITwo sub @ 60· Video-MME-v219.4%
48Qwen: Qwen3 VL 8B Thinkingw. sub @ 64, think· Video-MME-v219.2%
49Qwen3.5 35B A3Bw. sub @ 64, instruct· Video-MME-v218.4%
50Qwen3.5 27Bwo sub @ 64, think· Video-MME-v218.2%

Showing 1–50 of 78 results.