Skip to content

Benchmark

PutnamBench isabelle-wsolution solved

Math benchmarks measure how well models solve maths problems. Higher scores are better.

Models measured
2
Scores recorded
3
Sources
1
IQ weight
No weight published

Score distribution

215
Mean
4
Median
300.5
Std dev
1
10th percentile
640
90th percentile
1
Min
1640
The band covers the middle 80% of scores. The line marks the median.

Leaderboard

#ModelScore
1GPT 5.5Varto, isabelle-wsolution, avg $8/problem, Using GPT-5.5. All problems solved. 130 requires formalization fixes.· PutnamBench640
2GPT 4oDSP (GPT-4o), isabelle-wsolution, pass@10, pass@10 w/ T=0.1; 1971 B1, 1986 B1, 1995 B1, 2001 A1.· PutnamBench4
3GPT 4oGPT-4o, isabelle-wsolution, pass@10, pass@10 w/ T=0.7; 1988 B1.· PutnamBench1

Data credits: Trishul Lab, PutnamBench public submitted results. Model identities and scaffold budgets preserved; result licence unstated.