Benchmark
PutnamBench isabelle-wsolution solved
Math benchmarks measure how well models solve maths problems. Higher scores are better.
- Models measured
- 2
- Scores recorded
- 3
- Sources
- 1
- IQ weight
- No weight published
Score distribution
- 215
- Mean
- 4
- Median
- 300.5
- Std dev
- 1
- 10th percentile
- 640
- 90th percentile
- 1
- Min
Leaderboard
| # | Model | Score |
|---|---|---|
| 1 | GPT 5.5Varto, isabelle-wsolution, avg $8/problem, Using GPT-5.5. All problems solved. 130 requires formalization fixes.· PutnamBench | 640 |
| 2 | GPT 4oDSP (GPT-4o), isabelle-wsolution, pass@10, pass@10 w/ T=0.1; 1971 B1, 1986 B1, 1995 B1, 2001 A1.· PutnamBench | 4 |
| 3 | GPT 4oGPT-4o, isabelle-wsolution, pass@10, pass@10 w/ T=0.7; 1988 B1.· PutnamBench | 1 |
Data credits: Trishul Lab, PutnamBench public submitted results. Model identities and scaffold budgets preserved; result licence unstated.