Benchmark
SEED-Bench 2
Vision benchmarks measure how well models understand images. Scores are percentages.
- Models measured
- 12
- Scores recorded
- 12
- Sources
- 1
- IQ weight
- 1.27
Score distribution
- 60.2%
- Mean
- 59.9%
- Median
- 12.7%
- Std dev
- 52.7%
- 10th percentile
- 75%
- 90th percentile
- 25.1%
- Min
This benchmark has a weight of 1.27 for chat models, measured over 12 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.
Leaderboard
| # | Model | Score |
|---|---|---|
| 1 | GPT 4.1· OpenVLM Leaderboard data | 76% |
| 2 | GPT-4.1 mini· OpenVLM Leaderboard data | 75% |
| 3 | Gemini 1.5 Flash· OpenVLM Leaderboard data | 70.2% |
| 4 | internvl3-8b· OpenVLM Leaderboard data | 67.1% |
| 5 | GPT-4.1 nano· OpenVLM Leaderboard data | 64.4% |
| 6 | Gemma 3 27B IT· OpenVLM Leaderboard data | 59.9% |
| 7 | Molmo 7B-D· OpenVLM Leaderboard data | 58.2% |
| 8 | Phi 4 Multimodal Instruct· OpenVLM Leaderboard data | 58.1% |
| 9 | Pixtral 12B· OpenVLM Leaderboard data | 57.9% |
| 10 | Gemma 3 12B IT· OpenVLM Leaderboard data | 57.6% |
| 11 | Gemma 3 4B· OpenVLM Leaderboard data | 52.7% |
| 12 | kosmos-2· OpenVLM Leaderboard data | 25.1% |
Data credits: OpenVLM Leaderboard (OpenCompass / VLMEvalKit), Apache-2.0 (licence)