Skip to content

Benchmark

BixBench coverage

Science benchmarks measure how well models answer science questions. Scores are percentages. Higher is better.

Models measured
2
Scores recorded
6
Sources
1
IQ weight
No weight published

Score distribution

71.9%
Mean
100%
Median
39.9%
Std dev
21.5%
10th percentile
100%
90th percentile
9.8%
Min
9.8%100%
The band covers the middle 80% of scores. The line marks the median.

Leaderboard

#ModelScore
1Claude 3.5 Sonnetv1.5, zero-shot, mcq-refusal-False, coverage, latest· BixBench v1.5 baselines100%
2Claude 3.5 Sonnetv1.5, zero-shot, openended, coverage, latest· BixBench v1.5 baselines100%
3GPT 4ov1.5, zero-shot, mcq-refusal-False, coverage· BixBench v1.5 baselines100%
4GPT 4ov1.5, zero-shot, openended, coverage· BixBench v1.5 baselines100%
5Claude 3.5 Sonnetv1.5, zero-shot, mcq-refusal-True, coverage, latest· BixBench v1.5 baselines21.5%
6GPT 4ov1.5, zero-shot, mcq-refusal-True, coverage· BixBench v1.5 baselines9.8%

Data credits: FutureHouse and ScienceMachine, BixBench v1.5 public zero-shot baseline file. Apache-2.0 repository. (licence)