Benchmark
Aider Polyglot
Coding benchmarks compare models on programming tasks. Scores are percentages. Higher is better.
- Models measured
- 63
- Scores recorded
- 140
- Sources
- 5
- IQ weight
- 2.11
Score distribution
- 52.7%
- Mean
- 56.4%
- Median
- 24.3%
- Std dev
- 16.4%
- 10th percentile
- 81.3%
- 90th percentile
- 3.6%
- Min
This benchmark has a weight of 2.11 for chat models, measured over 63 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.
Leaderboard
| # | Model | Score |
|---|---|---|
| 101 | Grok 3 Minilow, beta· Epoch AI Benchmarking Hub data | 34.7% |
| 102 | GPT OSS 120Bofficial model card Table 3 · reasoning medium · published setting; Table 3 does not further specify tool access · pass@1· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 34.2% |
| 103 | GPT OSS 20bofficial model card Table 3 · reasoning high · published setting; Table 3 does not further specify tool access · pass@1· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 34.2% |
| 104 | O1 Mini· Epoch AI Benchmarking Hub data | 32.9% |
| 105 | O1 Miniaider· Aider LLM Leaderboards | 32.9% |
| 106 | GPT-4.1 miniaider· Aider LLM Leaderboards | 32.4% |
| 107 | GPT 4o· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claimValues from 2 sources
| 30.7% |
| 108 | Claude Haiku 3.5· Epoch AI Benchmarking Hub data | 28% |
| 109 | Claude Haiku 3.5aider· Aider LLM Leaderboards | 28% |
| 110 | ChatGPT-4o Latestaider· Aider LLM Leaderboards | 27.1% |
| 111 | Gemini 2.5 Flash-Lite· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim | 26.7% |
| 112 | GPT OSS 20bofficial model card Table 3 · reasoning medium · published setting; Table 3 does not further specify tool access · pass@1· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 26.6% |
| 113 | GPT OSS 120Bofficial model card Table 3 · reasoning low · published setting; Table 3 does not further specify tool access · pass@1· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 24% |
| 114 | GPT 4oaider· Aider LLM Leaderboards | 23.1% |
| 115 | Gemini 2.0 Flash Experimental· Epoch AI Benchmarking Hub data | 22.2% |
| 116 | Gemini 2.0 Flash Experimentalaider· Aider LLM Leaderboards | 22.2% |
| 117 | Qwen Max· Epoch AI Benchmarking Hub data | 21.8% |
| 118 | Qwen Maxaider· Aider LLM Leaderboards | 21.8% |
| 119 | QwQ 32B· Epoch AI Benchmarking Hub data | 20.9% |
| 120 | QwQ 32Baider· Aider LLM Leaderboards | 20.9% |
| 121 | Gemini 2.0 Flash Experimentalaider, thinking· Aider LLM Leaderboards | 18.2% |
| 122 | Gemini 2.0 Flash Experimentalthinking· Epoch AI Benchmarking Hub data | 18.2% |
| 123 | DeepSeek V2.5· Epoch AI Benchmarking Hub data | 17.8% |
| 124 | GPT OSS 20bofficial model card Table 3 · reasoning low · published setting; Table 3 does not further specify tool access · pass@1· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it. | 16.6% |
| 125 | Qwen 2.5 Coder 32B Instruct· Epoch AI Benchmarking Hub data | 16.4% |
| 126 | Llama 4 Maverick· Epoch AI Benchmarking Hub data | 15.6% |
| 127 | Llama 4 Maverickaider· Aider LLM Leaderboards | 15.6% |
| 128 | Yi Lightning· Epoch AI Benchmarking Hub data | 12.9% |
| 129 | Yi Lightningaider· Aider LLM Leaderboards | 12.9% |
| 130 | C4AI Command A 03 2025· Epoch AI Benchmarking Hub data | 12% |
| 131 | Codestral 25.012501· Epoch AI Benchmarking Hub data | 11.1% |
| 132 | openhands-lm-32b-v0.1· Epoch AI Benchmarking Hub data | 10.2% |
| 133 | openhands-lm-32b-v0.1aider· Aider LLM Leaderboards | 10.2% |
| 134 | GPT-4.1 nano· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claimValues from 2 sources
| 9.8% |
| 135 | GPT-4.1 nanoaider· Aider LLM Leaderboards | 8.9% |
| 136 | Qwen 2.5 Coder 32B Instructaider· Aider LLM Leaderboards | 8% |
| 137 | Gemma 3 27B IT· Epoch AI Benchmarking Hub data | 4.9% |
| 138 | Gemma 3 27B ITaider· Aider LLM Leaderboards | 4.9% |
| 139 | GPT-4o mini· Epoch AI Benchmarking Hub data | 3.6% |
| 140 | GPT-4o miniaider· Aider LLM Leaderboards | 3.6% |
Showing 101–140 of 140 results.
Data credits: Epoch AI Benchmarking Hub, CC BY 4.0 (licence) · Aider LLM Leaderboards (Aider-AI), Apache-2.0 (licence) · Steel.dev Agent Leaderboards, MIT (licence) · BenchBoard by SYuan03 (github.com/SYuan03/benchboard). MIT covers its code and data organization; no results licence is inferred. Numeric relays retain the original stating-source links. · Data by LLM Stats