Skip to content

Benchmark

LiveBench plot_unscrambling

Scores are percentages.

Models measured
51
Scores recorded
51
Sources
1
IQ weight
2.35

Score distribution

58.8%
Mean
58.6%
Median
10.9%
Std dev
43.9%
10th percentile
72.6%
90th percentile
22.9%
Min
22.9%81.9%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 2.35 for chat models, measured over 51 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1GPT-6.1 Solxhigh· LiveBench data81.9%
2Claude Opus 5.5xhigh· LiveBench data77.2%
3Gemini 3.1 Pro Preview· LiveBench data74.1%
4GPT 5.5xhigh· LiveBench data74.1%
5Gemini 3.8 Flashhigh· LiveBench data73.4%
6Kimi K3· LiveBench data72.6%
7Grok 4.7xhigh· LiveBench data72.4%
8Muse Spark 1.3xhigh· LiveBench data68.4%
9Claude Sonnet 5.5xhigh· LiveBench data67.8%
10Gemini 3.5 Flashhigh· LiveBench data67.8%
11Grok 4.6· LiveBench data67.1%
12Claude Opus 4.6auto, high· LiveBench data66.5%
13Claude Opus 4.564k, high· LiveBench data66.5%
14Grok 4.5· LiveBench data66.4%
15Gemini 3.7 Flashhigh· LiveBench data66.4%
16GPT 5.4xhigh· LiveBench data65.9%
17DeepSeek V4 Pro 0813· LiveBench data64.2%
18Gemini 3.6 Flashhigh· LiveBench data63.7%
19Muse Spark 1.1xhigh· LiveBench data63%
20Muse Spark 1.2xhigh· LiveBench data61.7%
21Claude Opus 4.7xhigh· LiveBench data61.4%
22GLM 5.2· LiveBench data60.7%
23GLM 5.3 Flash· LiveBench data60.3%
24GLM 5.3· LiveBench data60.2%
25Qwen3.7 Max· LiveBench data58.7%
26Qwen3.8 Max· LiveBench data58.6%
27GPT 5.2high· LiveBench data58.4%
28DeepSeek V4 Flash 0731· LiveBench data58.2%
29Kimi K2.6thinking· LiveBench data58.1%
30DeepSeek V4 Flash Vision Exp· LiveBench data57.7%
31Claude Sonnet 4.6auto, medium· LiveBench data57%
32Qwen 3.8 27B· LiveBench data56.5%
33DeepSeek V4 Pro· LiveBench data56.4%
34GPT-5.2 Codex· LiveBench data56%
35Qwen 3.8 Flash Next· LiveBench data55.8%
36Kimi K2.7 Code· LiveBench data55.7%
37Claude Sonnet 5xhigh· LiveBench data53.4%
38Grok Build 0.1· LiveBench data53.4%
39Inkling FP4xhigh· LiveBench data53.2%
40Qwen3.6 Plus· LiveBench data52.5%
41MiniMax M3· LiveBench data50.2%
42Gemini 3.5 Flash Litehigh· LiveBench data49.5%
43Grok 4.3· LiveBench data48.7%
44Nemotron 3 Ultra· LiveBench data47.8%
45DeepSeek V4 Flash· LiveBench data46.9%
46GPT-5.4 minixhigh· LiveBench data43.9%
47Qwen3.6 27B· LiveBench data42.7%
48Claude Haiku 5.5xhigh· LiveBench data42.7%
49Mistral Large 4high· LiveBench data40.9%
50GPT 5.4 Nanoxhigh· LiveBench data39.1%

Showing 1–50 of 51 results.