Skip to content

Benchmark

Video-MME-v2 - Level 2

Vision benchmarks measure how well models understand images. Scores are percentages.

Models measured
20
Scores recorded
78
Sources
1
IQ weight
1.25

Score distribution

24.2%
Mean
23.1%
Median
10.2%
Std dev
12.6%
10th percentile
40%
90th percentile
7.7%
Min
7.7%50%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.25 for chat models, measured over 20 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Gemini 3 Pro Previeww. sub @ 1fps· Video-MME-v250%
2Doubao Seed 2.0 Prow. sub @ 1fps· Video-MME-v247%
3Gemini 3 Pro Previewwo sub @ 1fps· Video-MME-v245.4%
4Gemini 3 Flashw. sub @ 1fps· Video-MME-v244.8%
5Kimi K2.5w. sub @ 512· Video-MME-v243.4%
6MiMo V2 Omniw. sub @ 1fps· Video-MME-v243.1%
7Doubao Seed 2.0 Prowo sub @ 1fps· Video-MME-v242.6%
8Qwen3.5 397B A17Bw. sub @ 512, think· Video-MME-v241.8%
9Kimi K2.5w. sub @ 64· Video-MME-v240%
10GPT 5w. sub @ 50· Video-MME-v239.1%
11Kimi K2.5wo sub @ 512· Video-MME-v237.8%
12Gemini 3 Flashwo sub @ 1fps· Video-MME-v237.4%
13Qwen3.5 397B A17Bwo sub @ 512, think· Video-MME-v236.5%
14Qwen3.5 27Bw. sub @ 512, think· Video-MME-v236.4%
15MiMo V2 Omniwo sub @ 1fps· Video-MME-v236%
16Kimi K2.5wo sub @ 64· Video-MME-v232.8%
17Qwen3.5 27Bw. sub @ 64, think· Video-MME-v230.6%
18Qwen3.5 397B A17Bw. sub @ 64, think· Video-MME-v230.4%
19Qwen3 VL 235Bw. sub @ 512, think· Video-MME-v230.3%
20Qwen3.5 9Bw. sub @ 512, think· Video-MME-v230.1%
21Gemma 4 31B ITw. sub @ 60· Video-MME-v229.2%
22GPT 5wo sub @ 50· Video-MME-v228.6%
23Qwen3.5 Omni Plusw. sub @ 512· Video-MME-v227.9%
24Qwen3.5 122B-A10Bw. sub @ 64, think· Video-MME-v227.9%
25Qwen3 VL 235Bw. sub @ 64, think· Video-MME-v227.5%
26Qwen3.5 Omni Plusw. sub @ 64· Video-MME-v226.6%
27Qwen3.5 397B A17Bw. sub @ 64, instruct· Video-MME-v225.9%
28Qwen3.5 122B-A10Bw. sub @ 512, think· Video-MME-v225.6%
29Qwen3 VL 235Bw. sub @ 64, instruct· Video-MME-v225.2%
30Qwen3.5 35B A3Bw. sub @ 64, think· Video-MME-v224.9%
31Qwen3.5 9Bw. sub @ 64, think· Video-MME-v224.7%
32Qwen3.5 35B A3Bw. sub @ 512, think· Video-MME-v224.5%
33Qwen3.5 9Bwo sub @ 512, think· Video-MME-v224.5%
34Qwen3.5 Omni Pluswo sub @ 512· Video-MME-v224.3%
35internvl3-5-241b-a28bw. sub @ 64, instruct· Video-MME-v223.7%
36Qwen3 VL 235Bwo sub @ 512, think· Video-MME-v223.4%
37Qwen3.5 27Bw. sub @ 64, instruct· Video-MME-v223.2%
38Qwen3.5 27Bwo sub @ 512, think· Video-MME-v223.1%
39Qwen3.5 397B A17Bwo sub @ 64, think· Video-MME-v223.1%
40qwen3-vl-30bw. sub @ 512, think· Video-MME-v222.4%
41Qwen3.5 122B-A10Bw. sub @ 64, instruct· Video-MME-v221.8%
42GLM 4.5Vw. sub @ 64, think· Video-MME-v221.8%
43Qwen: Qwen3 VL 8B Thinkingw. sub @ 512, think· Video-MME-v221.4%
44Qwen2.5 VL 72B Instructw. sub @ 64, instruct· Video-MME-v221.2%
45Gemma 4 31B ITwo sub @ 60· Video-MME-v221.1%
46qwen3-vl-30bw. sub @ 64, think· Video-MME-v221.1%
47Qwen3.5 4Bw. sub @ 64, think· Video-MME-v221%
48Qwen: Qwen3 VL 8B Thinkingw. sub @ 64, think· Video-MME-v219.2%
49Qwen3.5 35B A3Bw. sub @ 64, instruct· Video-MME-v218.6%
50Qwen3.5 27Bwo sub @ 64, think· Video-MME-v218.5%

Showing 1–50 of 78 results.