Skip to content

Benchmark

BIG-Bench Extra Hard

Scores are percentages.

Models measured
9
Scores recorded
9
Sources
1
IQ weight
1.59

Score distribution

33.4%
Mean
21.9%
Median
23.2%
Std dev
11%
10th percentile
64.8%
90th percentile
7.2%
Min
7.2%74.4%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.59 for chat models, measured over 9 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Gemma 4 31B IT· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim74.4%
2Gemma 4 26B A4B· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim64.8%
3Gemma 4 12B It· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim53%
4Gemma 4 E4B-it· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim33.1%
5Gemma 4 E2B-it· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim21.9%
6Gemma 3 27B IT· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim19.3%
7Gemma 3 12B IT· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim16.3%
8Gemma 3 4B IT· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim11%
9Gemma 3 1b it· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim7.2%

Data credits: Data by LLM Stats