Skip to content

Benchmark

ZEROBench pass@5

Vision benchmarks measure how well models understand images. Scores are percentages. Higher is better.

Models measured
62
Scores recorded
68
Sources
1
IQ weight
1.56

Score distribution

11%
Mean
7%
Median
10.6%
Std dev
1%
10th percentile
23%
90th percentile
0%
Min
0%52%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.56 for chat models, measured over 62 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1GPT-6 Astramax· ZeroBench52%
2Claude Opus 5.5max· ZeroBench43%
3GPT-6 Solmax· ZeroBench41%
4GPT-5.6 Solmax· ZeroBench30%
5Claude Opus 5max· ZeroBench26%
6Claude Fable 5max· ZeroBench24%
7GPT 5.4xhigh· ZeroBench23%
8GPT 5.5xhigh· ZeroBench23%
9Gemini 3.5 Flash· ZeroBench22%
10GPT-5.6 Lunamax· ZeroBench21%
11Gemini 3 Pro Preview· ZeroBench20%
12GPT-6 Lunamax· ZeroBench20%
13Gemini 3.1 Pro Preview· ZeroBench19%
14GPT-5.6 Terramax· ZeroBench19%
15Claude Opus 4.8· ZeroBench18%
16Gemini 3.6 Flashhigh· ZeroBench18%
17GPT 5.2medium reasoning· ZeroBench18%
18Grok 4.6xhigh· ZeroBench17%
19Claude Opus 4.7xhigh· ZeroBench15%
20Gemini 3 Flash· ZeroBench15%
21Grok 4.5high· ZeroBench15%
22Gemini 3.5 Flash Litehigh· ZeroBench14%
23Claude Sonnet 5max· ZeroBench13%
24Grok 4.3high· ZeroBench13%
25Claude Opus 4.6· ZeroBench12%
26Claude Sonnet 4.6· ZeroBench12%
27GPT-5.4 mini· ZeroBench11%
28o4 Mini· ZeroBench11%
29Grok 4.20non-reasoning· ZeroBench10%
30Grok 4.20reasoning· ZeroBench10%
31Claude Opus 4.5· ZeroBench9%
32GPT-5 minihigh reasoning· ZeroBench9%
33Claude Opus 4.1thinking· ZeroBench8%
34GPT 5medium reasoning· ZeroBench7%
35Claude Opus 4thinking· ZeroBench6%
36Claude Sonnet 3.7thinking· ZeroBench6%
37Claude Sonnet 4· ZeroBench6%
38Claude Sonnet 4thinking· ZeroBench6%
39GPT 5.1medium reasoning· ZeroBench6%
40GPT-4.5 Preview· ZeroBench6%
41Grok 4· ZeroBench6%
42OpenAI o3· ZeroBench6%
43Claude Opus 4.1· ZeroBench5%
44Claude Sonnet 4.5· ZeroBench5%
45Gemini 2.5 Pro· ZeroBench5%
46Gemini 3.1 Flash Lite· ZeroBench5%
47Claude Opus 4· ZeroBench4%
48Claude 3.5 Sonnet v2· ZeroBench3%
49Claude Sonnet 4.5thinking· ZeroBench3%
50Gemini 2.5 Flash· ZeroBench3%

Showing 1–50 of 68 results.