Skip to content

Benchmark

Video-MME-v2 - Avg Acc

Vision benchmarks measure how well models understand images. Scores are percentages.

Models measured
20
Scores recorded
78
Sources
1
IQ weight
1.40

Score distribution

40.1%
Mean
38.9%
Median
9.6%
Std dev
28.9%
10th percentile
54.4%
90th percentile
21.3%
Min
21.3%66.1%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.40 for chat models, measured over 20 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Gemini 3 Pro Previeww. sub @ 1fps· Video-MME-v266.1%
2Gemini 3 Flashw. sub @ 1fps· Video-MME-v261.1%
3Doubao Seed 2.0 Prow. sub @ 1fps· Video-MME-v260.5%
4Kimi K2.5w. sub @ 512· Video-MME-v258%
5Gemini 3 Pro Previewwo sub @ 1fps· Video-MME-v256.8%
6MiMo V2 Omniw. sub @ 1fps· Video-MME-v256.1%
7Qwen3.5 397B A17Bw. sub @ 512, think· Video-MME-v255.9%
8GPT 5w. sub @ 50· Video-MME-v255.6%
9Kimi K2.5w. sub @ 64· Video-MME-v254.4%
10Doubao Seed 2.0 Prowo sub @ 1fps· Video-MME-v253.1%
11Gemini 3 Flashwo sub @ 1fps· Video-MME-v252.4%
12Kimi K2.5wo sub @ 512· Video-MME-v251.2%
13Qwen3.5 27Bw. sub @ 512, think· Video-MME-v249.6%
14Qwen3.5 397B A17Bw. sub @ 64, think· Video-MME-v248.9%
15Qwen3.5 397B A17Bwo sub @ 512, think· Video-MME-v248.8%
16Qwen3.5 27Bw. sub @ 64, think· Video-MME-v247.6%
17Qwen3 VL 235Bw. sub @ 512, think· Video-MME-v247.2%
18MiMo V2 Omniwo sub @ 1fps· Video-MME-v247.1%
19Kimi K2.5wo sub @ 64· Video-MME-v246%
20Qwen3.5 Omni Plusw. sub @ 512· Video-MME-v245.8%
21Gemma 4 31B ITw. sub @ 60· Video-MME-v245.6%
22Qwen3.5 122B-A10Bw. sub @ 64, think· Video-MME-v245.1%
23Qwen3 VL 235Bw. sub @ 64, think· Video-MME-v244.9%
24GPT 5wo sub @ 50· Video-MME-v244.7%
25Qwen3.5 9Bw. sub @ 512, think· Video-MME-v244.5%
26Qwen3.5 Omni Plusw. sub @ 64· Video-MME-v244.3%
27Qwen3.5 35B A3Bw. sub @ 64, think· Video-MME-v243.9%
28Qwen3.5 122B-A10Bw. sub @ 512, think· Video-MME-v243.4%
29Qwen3 VL 235Bw. sub @ 64, instruct· Video-MME-v243.3%
30Qwen3.5 397B A17Bw. sub @ 64, instruct· Video-MME-v242.1%
31Qwen3.5 27Bw. sub @ 64, instruct· Video-MME-v241.9%
32Qwen3.5 35B A3Bw. sub @ 512, think· Video-MME-v241.5%
33internvl3-5-241b-a28bw. sub @ 64, instruct· Video-MME-v241.4%
34Qwen3.5 9Bw. sub @ 64, think· Video-MME-v240.3%
35qwen3-vl-30bw. sub @ 512, think· Video-MME-v239.8%
36Qwen3.5 397B A17Bwo sub @ 64, think· Video-MME-v239.3%
37GLM 4.5Vw. sub @ 64, think· Video-MME-v239.2%
38Qwen3.5 122B-A10Bw. sub @ 64, instruct· Video-MME-v239.2%
39Qwen2.5 VL 72B Instructw. sub @ 64, instruct· Video-MME-v238.9%
40Qwen3.5 4Bw. sub @ 64, think· Video-MME-v238.8%
41qwen3-vl-30bw. sub @ 64, think· Video-MME-v238.6%
42Qwen3.5 Omni Pluswo sub @ 512· Video-MME-v238.2%
43Qwen3.5 27Bwo sub @ 512, think· Video-MME-v237.4%
44Qwen: Qwen3 VL 8B Thinkingw. sub @ 512, think· Video-MME-v237.3%
45Qwen: Qwen3 VL 8B Thinkingw. sub @ 64, think· Video-MME-v237.2%
46Qwen3.5 35B A3Bw. sub @ 64, instruct· Video-MME-v236.9%
47Qwen3 VL 235Bwo sub @ 512, think· Video-MME-v236.8%
48Qwen3.5 9Bwo sub @ 512, think· Video-MME-v236.8%
49Gemma 4 31B ITwo sub @ 60· Video-MME-v236.7%
50Qwen: Qwen3 VL 8B Thinkingw. sub @ 64, instruct· Video-MME-v235.7%

Showing 1–50 of 78 results.