Skip to content

Benchmark

BixBench precision

Science benchmarks measure how well models answer science questions. Scores are percentages. Higher is better.

Models measured
2
Scores recorded
6
Sources
1
IQ weight
No weight published

Score distribution

25.8%
Mean
36.1%
Median
16.3%
Std dev
2.9%
10th percentile
40%
90th percentile
2.9%
Min
2.9%40%
The band covers the middle 80% of scores. The line marks the median.

Leaderboard

#ModelScore
1GPT 4ov1.5, zero-shot, mcq-refusal-True, precision· BixBench v1.5 baselines40%
2Claude 3.5 Sonnetv1.5, zero-shot, mcq-refusal-True, precision, latest· BixBench v1.5 baselines38.6%
3GPT 4ov1.5, zero-shot, mcq-refusal-False, precision· BixBench v1.5 baselines36.1%
4Claude 3.5 Sonnetv1.5, zero-shot, mcq-refusal-False, precision, latest· BixBench v1.5 baselines34.1%
5Claude 3.5 Sonnetv1.5, zero-shot, openended, precision, latest· BixBench v1.5 baselines2.9%
6GPT 4ov1.5, zero-shot, openended, precision· BixBench v1.5 baselines2.9%

Data credits: FutureHouse and ScienceMachine, BixBench v1.5 public zero-shot baseline file. Apache-2.0 repository. (licence)