Skip to content

Benchmark

Video-MME-v2 - Level 3

Vision benchmarks measure how well models understand images. Scores are percentages.

Models measured
20
Scores recorded
78
Sources
1
IQ weight
1.63

Score distribution

19%
Mean
18.4%
Median
6.8%
Std dev
10.9%
10th percentile
28.4%
90th percentile
7.1%
Min
7.1%40.5%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.63 for chat models, measured over 20 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Gemini 3 Pro Previeww. sub @ 1fps· Video-MME-v240.5%
2Doubao Seed 2.0 Prow. sub @ 1fps· Video-MME-v234.1%
3Kimi K2.5w. sub @ 512· Video-MME-v232.2%
4Gemini 3 Flashw. sub @ 1fps· Video-MME-v231.7%
5GPT 5w. sub @ 50· Video-MME-v231.1%
6Qwen3.5 397B A17Bw. sub @ 512, think· Video-MME-v230.7%
7Gemini 3 Pro Previewwo sub @ 1fps· Video-MME-v230.2%
8Kimi K2.5w. sub @ 64· Video-MME-v228.5%
9Qwen3.5 397B A17Bw. sub @ 64, think· Video-MME-v228.4%
10MiMo V2 Omniw. sub @ 1fps· Video-MME-v227.4%
11Doubao Seed 2.0 Prowo sub @ 1fps· Video-MME-v226.2%
12Qwen3.5 27Bw. sub @ 512, think· Video-MME-v226.2%
13Gemma 4 31B ITw. sub @ 60· Video-MME-v226.1%
14Qwen3.5 27Bw. sub @ 64, think· Video-MME-v225.8%
15Gemini 3 Flashwo sub @ 1fps· Video-MME-v225%
16Kimi K2.5wo sub @ 512· Video-MME-v224.8%
17Qwen3.5 122B-A10Bw. sub @ 64, think· Video-MME-v223.9%
18Qwen3 VL 235Bw. sub @ 512, think· Video-MME-v223.9%
19Qwen3.5 122B-A10Bw. sub @ 512, think· Video-MME-v223.4%
20Qwen3.5 35B A3Bw. sub @ 64, think· Video-MME-v223.2%
21Qwen3.5 397B A17Bwo sub @ 512, think· Video-MME-v222.9%
22Qwen3 VL 235Bw. sub @ 64, think· Video-MME-v222.3%
23Qwen3.5 397B A17Bw. sub @ 64, instruct· Video-MME-v221.9%
24Kimi K2.5wo sub @ 64· Video-MME-v221.7%
25Qwen3 VL 235Bw. sub @ 64, instruct· Video-MME-v221.6%
26Qwen3.5 Omni Plusw. sub @ 64· Video-MME-v221.6%
27Qwen3.5 27Bw. sub @ 64, instruct· Video-MME-v221.5%
28GPT 5wo sub @ 50· Video-MME-v221.3%
29Qwen3.5 Omni Plusw. sub @ 512· Video-MME-v221.2%
30Qwen3.5 9Bw. sub @ 512, think· Video-MME-v220.9%
31Qwen2.5 VL 72B Instructw. sub @ 64, instruct· Video-MME-v220.6%
32MiMo V2 Omniwo sub @ 1fps· Video-MME-v220.4%
33Qwen3.5 35B A3Bw. sub @ 512, think· Video-MME-v220.1%
34internvl3-5-241b-a28bw. sub @ 64, instruct· Video-MME-v219.6%
35GLM 4.5Vw. sub @ 64, think· Video-MME-v219.4%
36Qwen3.5 9Bw. sub @ 64, think· Video-MME-v219%
37Qwen3.5 397B A17Bwo sub @ 64, think· Video-MME-v219%
38qwen3-vl-30bw. sub @ 512, think· Video-MME-v218.7%
39Qwen3.5 35B A3Bw. sub @ 64, instruct· Video-MME-v218.4%
40Qwen3.5 122B-A10Bw. sub @ 64, instruct· Video-MME-v217.9%
41qwen3-vl-30bw. sub @ 64, think· Video-MME-v217.6%
42Qwen: Qwen3 VL 8B Thinkingw. sub @ 64, think· Video-MME-v217.5%
43Qwen3.5 9Bw. sub @ 64, instruct· Video-MME-v217.4%
44Qwen: Qwen3 VL 8B Thinkingw. sub @ 512, think· Video-MME-v217.4%
45Qwen3.5 4Bw. sub @ 64, think· Video-MME-v217.3%
46Qwen: Qwen3 VL 8B Thinkingw. sub @ 64, instruct· Video-MME-v216.9%
47Qwen3.5 397B A17Bwo sub @ 64, instruct· Video-MME-v216.1%
48Gemma 4 31B ITwo sub @ 60· Video-MME-v216%
49qwen3-vl-30bw. sub @ 64, instruct· Video-MME-v215.6%
50Qwen3.5 27Bwo sub @ 64, think· Video-MME-v215.5%

Showing 1–50 of 78 results.