Benchmark
Aider Polyglot
Coding benchmarks compare models on programming tasks. Scores are percentages. Higher is better.
- Models measured
- 63
- Scores recorded
- 140
- Sources
- 5
- IQ weight
- 2.11
Score distribution
- 52.7%
- Mean
- 56.4%
- Median
- 24.3%
- Std dev
- 16.4%
- 10th percentile
- 81.3%
- 90th percentile
- 3.6%
- Min
This benchmark has a weight of 2.11 for chat models, measured over 63 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.
Leaderboard
| # | Model | Score |
|---|---|---|
| 1 | Claude Opus 4.5launch-page coding chart· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 89.4% |
| 2 | GPT 5· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim | 88% |
| 3 | GPT 5aider-reasoning-high, high· Aider LLM Leaderboards | 88% |
| 4 | GPT 5high· Epoch AI Benchmarking Hub dataValues from 2 sources
| 88% |
| 5 | GPT 5aider-reasoning-medium, medium· Aider LLM Leaderboards | 86.7% |
| 6 | GPT 5medium· Epoch AI Benchmarking Hub data | 86.7% |
| 7 | O3 Proaider-reasoning-high, high· Aider LLM Leaderboards | 84.9% |
| 8 | O3 Prohigh· Epoch AI Benchmarking Hub dataValues from 2 sources
| 84.9% |
| 9 | Gemini 2.5 Pro32k· Epoch AI Benchmarking Hub data | 83.1% |
| 10 | Gemini 2.5 Pro32k, think· Steel.dev Agent Leaderboards data | 83.1% |
| 11 | Gemini 2.5 Proaider-reasoning-32k, 32k think· Aider LLM Leaderboards | 83.1% |
| 12 | Gemini 2.5 Pro· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claimValues from 2 sources
| 82.2% |
| 13 | GPT 5aider-reasoning-low, low· Aider LLM Leaderboards | 81.3% |
| 14 | GPT 5low· Epoch AI Benchmarking Hub dataValues from 2 sources
| 81.3% |
| 15 | OpenAI o3· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claimValues from 3 sources
| 81.3% |
| 16 | OpenAI o3aider-reasoning-high, high· Aider LLM Leaderboards | 81.3% |
| 17 | OpenAI o3high· Epoch AI Benchmarking Hub dataValues from 2 sources
| 81.3% |
| 18 | Grok 4· Epoch AI Benchmarking Hub data | 79.6% |
| 19 | Grok 4aider-reasoning-high, high· Aider LLM Leaderboards | 79.6% |
| 20 | Grok 4high· Steel.dev Agent Leaderboards data | 79.6% |
| 21 | Gemini 2.5 Proaider-reasoning-default, default think· Aider LLM Leaderboards | 79.1% |
| 22 | Gemini 2.5 Prothink· Steel.dev Agent Leaderboards data | 79.1% |
| 23 | Claude Sonnet 4.5launch-page coding chart· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 78.8% |
| 24 | Gemini 2.5 Pro Preview 05-06· Epoch AI Benchmarking Hub dataValues from 2 sources
| 76.9% |
| 25 | Gemini 2.5 Pro Preview 05-06aider· Aider LLM Leaderboards | 76.9% |
| 26 | OpenAI o3aider· Aider LLM Leaderboards | 76.9% |
| 27 | OpenAI o3medium· Epoch AI Benchmarking Hub data | 76.9% |
| 28 | DeepSeek V3.2 Exp· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claimValues from 2 sources
| 74.5% |
| 29 | DeepSeek Reasoner· Epoch AI Benchmarking Hub data | 74.2% |
| 30 | DeepSeek V3.2 Expthinking· Epoch AI Benchmarking Hub data | 74.2% |
| 31 | Gemini 2.5 Pro· Epoch AI Benchmarking Hub data | 72.9% |
| 32 | Gemini 2.5 Pro Preview 03-25· Epoch AI Benchmarking Hub dataValues from 2 sources
| 72.9% |
| 33 | Gemini 2.5 Pro Preview 03-25aider· Aider LLM Leaderboards | 72.9% |
| 34 | Claude Opus 432k· Epoch AI Benchmarking Hub data | 72% |
| 35 | Claude Opus 432k, thinking· Steel.dev Agent Leaderboards data | 72% |
| 36 | Claude Opus 4aider-reasoning-32k, 32k thinking· Aider LLM Leaderboards | 72% |
| 37 | o4 Mini· Steel.dev Agent Leaderboards dataValues from 2 sources
| 72% |
| 38 | o4 Miniaider-reasoning-high, high· Aider LLM Leaderboards | 72% |
| 39 | o4 Minihigh· Epoch AI Benchmarking Hub data | 72% |
| 40 | DeepSeek R1· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claimValues from 2 sources
| 71.6% |
| 41 | Claude Opus 4· Epoch AI Benchmarking Hub data | 70.7% |
| 42 | Claude Opus 4aider-reasoning-off, no think· Aider LLM Leaderboards | 70.7% |
| 43 | DeepSeek Chat· Epoch AI Benchmarking Hub data | 70.2% |
| 44 | DeepSeek V3.2 Expaider, Chat· Aider LLM Leaderboards | 70.2% |
| 45 | DeepSeek V3.1· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim | 68.4% |
| 46 | o3 Mini· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim | 66.7% |
| 47 | Claude Sonnet 3.732k· Epoch AI Benchmarking Hub data | 64.9% |
| 48 | Claude Sonnet 3.7aider-reasoning-32k, 32k thinking tokens· Aider LLM Leaderboards | 64.9% |
| 49 | Gemini 2.5 Flash· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim | 61.9% |
| 50 | Qwen3 Coder· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim | 61.8% |
Showing 1–50 of 140 results.
Data credits: Epoch AI Benchmarking Hub, CC BY 4.0 (licence) · Aider LLM Leaderboards (Aider-AI), Apache-2.0 (licence) · Steel.dev Agent Leaderboards, MIT (licence) · BenchBoard by SYuan03 (github.com/SYuan03/benchboard). MIT covers its code and data organization; no results licence is inferred. Numeric relays retain the original stating-source links. · Data by LLM Stats