Skip to content

Benchmark

Judgemark v2

Scores are percentages.

Scores from EQ-Bench boards.
Models measured
46
Scores recorded
46
Sources
1
IQ weight
2.17

Score distribution

70%
Mean
75%
Median
15.5%
Std dev
52.2%
10th percentile
85.5%
90th percentile
19.2%
Min
19.2%94.1%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 2.17 for chat models, measured over 46 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Claude Sonnet 4.6· EQ-Bench boards94.1%
2Claude Opus 4.6· EQ-Bench boards92.9%
3Gemini 3 Pro Preview· EQ-Bench boards86.6%
4GLM 5· EQ-Bench boards85.6%
5Qwen3.5 397B A17B· EQ-Bench boards85.5%
6Gemini 3 Flash· EQ-Bench boards85.4%
7GPT 5.4· EQ-Bench boards85%
8Claude Sonnet 4.5· EQ-Bench boards84.6%
9Qwen3.5 27B· EQ-Bench boards84.3%
10GPT 5.2· EQ-Bench boards82%
11Claude Sonnet 4· EQ-Bench boards82%
12Qwen: Qwen3.5-Flash· EQ-Bench boards81.8%
13Qwen3.5 35B A3B· EQ-Bench boards81.4%
14GPT 5minimal, reasoning· EQ-Bench boards81.2%
15GPT-5.4 mini· EQ-Bench boards81.1%
16Qwen3.5 122B-A10B· EQ-Bench boards81%
17Gemini 3.1 Flash Lite· EQ-Bench boards80.9%
18Claude Haiku 4.5· EQ-Bench boards79.7%
19GLM 4.7· EQ-Bench boards77.5%
20GPT 4.1· EQ-Bench boards76.3%
21Grok 4.1 Fast· EQ-Bench boards75.8%
22OpenAI o3· EQ-Bench boards75.1%
23Qwen3.5 9B· EQ-Bench boards75%
24GPT-5 miniminimal, reasoning· EQ-Bench boards74.5%
25Kimi K2 Instruct· EQ-Bench boards73.1%
26Gemini 2.5 Pro· EQ-Bench boards72.1%
27Mistral Large 3 675b Instruct· EQ-Bench boards68.2%
28DeepSeek V3.2· EQ-Bench boards66.3%
29GLM 4.7 Flash· EQ-Bench boards66.2%
30GPT 5.4 Nano· EQ-Bench boards64.5%
31MiniMax M2.5· EQ-Bench boards64.5%
32GLM 4.5· EQ-Bench boards63.4%
33Gemini 2.5 Flash· EQ-Bench boards63%
34Qwen3 235B A22B Instruct 2507· EQ-Bench boards59.7%
35Mistral Medium 3· EQ-Bench boards58.5%
36GPT-4.1 mini· EQ-Bench boards57.3%
37Qwen 3 235B A22B· EQ-Bench boards56.2%
38o4 Mini· EQ-Bench boards56.1%
39Gemini 2.5 Flash-Lite· EQ-Bench boards56%
40GPT OSS 120B· EQ-Bench boards54.8%
41GPT-5 nanominimal, reasoning· EQ-Bench boards52.2%
42Mistral Small 4 119B· EQ-Bench boards51.6%
43Mistral Small 3.2 24B Instruct· EQ-Bench boards50.2%
44GPT OSS 20b· EQ-Bench boards42.8%
45Qwen3 30B A3B Instruct· EQ-Bench boards34.6%
46Llama 3.1 8B Instruct· EQ-Bench boards19.2%

Data credits: EQ-Bench (Samuel J. Paech) — cite paech2023eqbench