Skip to content

Benchmark

Shaberi v2.1 (GPT5.1 judge): ELYZA 100

People compare model answers and choose which they prefer. Higher scores are better.

Models measured
44
Scores recorded
47
Sources
1
IQ weight
1.13

Score distribution

6.8
Mean
7.3
Median
2
Std dev
3.5
10th percentile
8.9
90th percentile
2
Min
29.3
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.13 for chat models, measured over 44 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Gemini 3 Flash· Shaberi v2.1 (Shisa.AI) data docs9.3
2Gemini 3 Pro Preview· Shaberi v2.1 (Shisa.AI) data docs9.3
3Gemini 2.5 Flash· Shaberi v2.1 (Shisa.AI) data docs9.2
4Gemini 2.5 Pro· Shaberi v2.1 (Shisa.AI) data docs9.1
5Qwen3.6 Plusfree· Shaberi v2.1 (Shisa.AI) data docs9
6Qwen3.5 397B A17B· Shaberi v2.1 (Shisa.AI) data docs8.9
7Qwen3 235B A22B Instruct 2507· Shaberi v2.1 (Shisa.AI) data docs8.9
8GPT 4o· Shaberi v2.1 (Shisa.AI) data docs8.8
9Gemma 4 31B IT· Shaberi v2.1 (Shisa.AI) data docs8.7
10Kimi K2.5· Shaberi v2.1 (Shisa.AI) data docs8.6
11Gemma 4 26B A4B· Shaberi v2.1 (Shisa.AI) data docs8.5
12DeepSeek V3.1 Terminus· Shaberi v2.1 (Shisa.AI) data docs8.4
13GPT OSS 120B· Shaberi v2.1 (Shisa.AI) data docs8.4
14Kimi K2 Instruct· Shaberi v2.1 (Shisa.AI) data docs8.3
15Llama 3.1 405B Instruct· Shaberi v2.1 (Shisa.AI) data docs8.3
16Qwen3 30B A3B Instruct· Shaberi v2.1 (Shisa.AI) data docs8.1
17Gemma 4 E4B-it· Shaberi v2.1 (Shisa.AI) data docs8
18Ling 1t· Shaberi v2.1 (Shisa.AI) data docs8
19Qwen3.6 27Bnothink· Shaberi v2.1 (Shisa.AI) data docs7.9
20Ministral 3 14B Instruct· Shaberi v2.1 (Shisa.AI) data docs7.9
21Llama 3.3 70B Instruct· Shaberi v2.1 (Shisa.AI) data docs7.7
22Gemma 3 27B IT· Shaberi v2.1 (Shisa.AI) data docs7.7
23Nemotron 3 Nano 30B A3B· Shaberi v2.1 (Shisa.AI) data docs7.6
24Ministral 3 8b Instruct· Shaberi v2.1 (Shisa.AI) data docs7.3
25GPT OSS 20b· Shaberi v2.1 (Shisa.AI) data docs7.2
26GLM 4.7 Flash· Shaberi v2.1 (Shisa.AI) data docs6.9
27Ling Flash 2.0· Shaberi v2.1 (Shisa.AI) data docs6.9
28Phi-4-reasoning· Shaberi v2.1 (Shisa.AI) data docs6.8
29Qwen3 8B· Shaberi v2.1 (Shisa.AI) data docs6.8
30Phi-4-reasoningnothink· Shaberi v2.1 (Shisa.AI) data docs6.8
31ERNIE-4.5-21B-A3B-PT· Shaberi v2.1 (Shisa.AI) data docs6.7
32Llama 3.1 8B Instruct· Shaberi v2.1 (Shisa.AI) data docs6.5
33Ministral 3 3b Instruct· Shaberi v2.1 (Shisa.AI) data docs6.1
34Phi 4 Multimodal Instruct· Shaberi v2.1 (Shisa.AI) data docs6
35ERNIE-4.5-VL-28B-A3B-PT· Shaberi v2.1 (Shisa.AI) data docs5.9
36Qwen3.5 122B-A10B· Shaberi v2.1 (Shisa.AI) data docs5.6
37Qwen3.6 27B32k· Shaberi v2.1 (Shisa.AI) data docs5.5
38Phi 4 Mini· Shaberi v2.1 (Shisa.AI) data docs5.4
39Llama 4 Scout· Shaberi v2.1 (Shisa.AI) data docs5.1
40Llama 3.2 3B· Shaberi v2.1 (Shisa.AI) data docs5.1
41Olmo 3 7B Instruct· Shaberi v2.1 (Shisa.AI) data docs4.5
42Nemotron 3 Nano 30B A3Bnothink· Shaberi v2.1 (Shisa.AI) data docs3.5
43Llama 3.2 1B Instruct· Shaberi v2.1 (Shisa.AI) data docs3.4
44Mistral (7B) Instruct v0.1· Shaberi v2.1 (Shisa.AI) data docs3.3
45Qwen3.5 27B· Shaberi v2.1 (Shisa.AI) data docs2
46Qwen3.5 35B A3B· Shaberi v2.1 (Shisa.AI) data docs2
47Qwen3.5 9B· Shaberi v2.1 (Shisa.AI) data docs2

Data credits: Shaberi v2.1, Shisa.AI, Apache-2.0. GPT-5.1-judge scores only; judges are not comparable. (licence)