Skip to content

Benchmark

Video-MME-v2 - Level 1

Vision benchmarks measure how well models understand images. Scores are percentages.

Models measured
20
Scores recorded
78
Sources
1
IQ weight
2.15

Score distribution

27.2%
Mean
25.8%
Median
11.7%
Std dev
14.9%
10th percentile
43.2%
90th percentile
9.3%
Min
9.3%64%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 2.15 for chat models, measured over 20 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Gemini 3 Pro Previeww. sub @ 1fps· Video-MME-v264%
2Gemini 3 Flashw. sub @ 1fps· Video-MME-v258.3%
3Doubao Seed 2.0 Prow. sub @ 1fps· Video-MME-v254.5%
4MiMo V2 Omniw. sub @ 1fps· Video-MME-v252.6%
5Kimi K2.5w. sub @ 512· Video-MME-v251.8%
6Qwen3.5 397B A17Bw. sub @ 512, think· Video-MME-v250.3%
7GPT 5w. sub @ 50· Video-MME-v244.5%
8Kimi K2.5w. sub @ 64· Video-MME-v244.3%
9Gemini 3 Pro Previewwo sub @ 1fps· Video-MME-v243.2%
10Gemini 3 Flashwo sub @ 1fps· Video-MME-v241.5%
11Doubao Seed 2.0 Prowo sub @ 1fps· Video-MME-v241.3%
12Kimi K2.5wo sub @ 512· Video-MME-v241.1%
13MiMo V2 Omniwo sub @ 1fps· Video-MME-v238.7%
14Qwen3.5 Omni Plusw. sub @ 512· Video-MME-v237.4%
15Qwen3.5 397B A17Bwo sub @ 512, think· Video-MME-v235.4%
16Qwen3.5 27Bw. sub @ 64, think· Video-MME-v235%
17Qwen3.5 397B A17Bw. sub @ 64, think· Video-MME-v234.8%
18Qwen3.5 27Bw. sub @ 512, think· Video-MME-v234.4%
19Qwen3 VL 235Bw. sub @ 512, think· Video-MME-v232.6%
20GPT 5wo sub @ 50· Video-MME-v232.2%
21Qwen3 VL 235Bw. sub @ 64, think· Video-MME-v232%
22Qwen3.5 35B A3Bw. sub @ 64, think· Video-MME-v231.3%
23Qwen3.5 Omni Plusw. sub @ 64· Video-MME-v231.2%
24Qwen3 VL 235Bw. sub @ 64, instruct· Video-MME-v230.7%
25Gemma 4 31B ITw. sub @ 60· Video-MME-v230.5%
26Qwen3.5 9Bw. sub @ 512, think· Video-MME-v230.5%
27Kimi K2.5wo sub @ 64· Video-MME-v230.5%
28Qwen3.5 122B-A10Bw. sub @ 64, think· Video-MME-v230.1%
29Qwen3.5 35B A3Bw. sub @ 512, think· Video-MME-v229.5%
30Qwen3.5 27Bw. sub @ 64, instruct· Video-MME-v228.9%
31Qwen3.5 9Bw. sub @ 64, think· Video-MME-v228.3%
32internvl3-5-241b-a28bw. sub @ 64, instruct· Video-MME-v228.2%
33qwen3-vl-30bw. sub @ 512, think· Video-MME-v227.6%
34Qwen3.5 122B-A10Bw. sub @ 512, think· Video-MME-v227.4%
35Qwen3.5 397B A17Bw. sub @ 64, instruct· Video-MME-v227.3%
36qwen3-vl-30bw. sub @ 64, think· Video-MME-v227%
37Qwen3.5 4Bw. sub @ 64, think· Video-MME-v226.2%
38GLM 4.5Vw. sub @ 64, think· Video-MME-v226.2%
39Qwen2.5 VL 72B Instructw. sub @ 64, instruct· Video-MME-v225.8%
40Qwen3.5 122B-A10Bw. sub @ 64, instruct· Video-MME-v224.9%
41Qwen3.5 35B A3Bw. sub @ 64, instruct· Video-MME-v224.2%
42Qwen: Qwen3 VL 8B Thinkingw. sub @ 64, think· Video-MME-v223.9%
43Qwen: Qwen3 VL 8B Thinkingw. sub @ 512, think· Video-MME-v223.5%
44Qwen3.5 27Bwo sub @ 512, think· Video-MME-v222.3%
45Qwen: Qwen3 VL 8B Thinkingw. sub @ 64, instruct· Video-MME-v221.8%
46Qwen3.5 9Bwo sub @ 512, think· Video-MME-v221.8%
47Gemma 4 31B ITwo sub @ 60· Video-MME-v221.5%
48Qwen3 VL 235Bwo sub @ 512, think· Video-MME-v221.4%
49Qwen3.5 Omni Pluswo sub @ 512· Video-MME-v221.3%
50Qwen3.5 397B A17Bwo sub @ 64, think· Video-MME-v220.7%

Showing 1–50 of 78 results.