Skip to content

Benchmark

BixBench accuracy

Science benchmarks measure how well models answer science questions. Scores are percentages. Higher is better.

Models measured
2
Scores recorded
6
Sources
1
IQ weight
No weight published

Score distribution

14.7%
Mean
8.3%
Median
14.6%
Std dev
2.9%
10th percentile
36.1%
90th percentile
2.9%
Min

Leaderboard

#ModelScore
1GPT 4ov1.5, zero-shot, mcq-refusal-False, accuracy· BixBench v1.5 baselines36.1%
2Claude 3.5 Sonnetv1.5, zero-shot, mcq-refusal-False, accuracy, latest· BixBench v1.5 baselines34.1%
3Claude 3.5 Sonnetv1.5, zero-shot, mcq-refusal-True, accuracy, latest· BixBench v1.5 baselines8.3%
4GPT 4ov1.5, zero-shot, mcq-refusal-True, accuracy· BixBench v1.5 baselines3.9%
5Claude 3.5 Sonnetv1.5, zero-shot, openended, accuracy, latest· BixBench v1.5 baselines2.9%
6GPT 4ov1.5, zero-shot, openended, accuracy· BixBench v1.5 baselines2.9%

Data credits: FutureHouse and ScienceMachine, BixBench v1.5 public zero-shot baseline file. Apache-2.0 repository. (licence)