Skip to content

Benchmark

Aider Polyglot

Coding benchmarks compare models on programming tasks. Scores are percentages. Higher is better.

Models measured
63
Scores recorded
140
Sources
5
IQ weight
2.11

Score distribution

52.7%
Mean
56.4%
Median
24.3%
Std dev
16.4%
10th percentile
81.3%
90th percentile
3.6%
Min
3.6%89.4%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 2.11 for chat models, measured over 63 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
101Grok 3 Minilow, beta· Epoch AI Benchmarking Hub data34.7%
102GPT OSS 120Bofficial model card Table 3 · reasoning medium · published setting; Table 3 does not further specify tool access · pass@1· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.34.2%
103GPT OSS 20bofficial model card Table 3 · reasoning high · published setting; Table 3 does not further specify tool access · pass@1· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.34.2%
104O1 Mini· Epoch AI Benchmarking Hub data32.9%
105O1 Miniaider· Aider LLM Leaderboards32.9%
106GPT-4.1 miniaider· Aider LLM Leaderboards32.4%
107GPT 4o· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim
Values from 2 sources
30.7%
108Claude Haiku 3.5· Epoch AI Benchmarking Hub data28%
109Claude Haiku 3.5aider· Aider LLM Leaderboards28%
110ChatGPT-4o Latestaider· Aider LLM Leaderboards27.1%
111Gemini 2.5 Flash-Lite· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim26.7%
112GPT OSS 20bofficial model card Table 3 · reasoning medium · published setting; Table 3 does not further specify tool access · pass@1· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.26.6%
113GPT OSS 120Bofficial model card Table 3 · reasoning low · published setting; Table 3 does not further specify tool access · pass@1· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.24%
114GPT 4oaider· Aider LLM Leaderboards23.1%
115Gemini 2.0 Flash Experimental· Epoch AI Benchmarking Hub data22.2%
116Gemini 2.0 Flash Experimentalaider· Aider LLM Leaderboards22.2%
117Qwen Max· Epoch AI Benchmarking Hub data21.8%
118Qwen Maxaider· Aider LLM Leaderboards21.8%
119QwQ 32B· Epoch AI Benchmarking Hub data20.9%
120QwQ 32Baider· Aider LLM Leaderboards20.9%
121Gemini 2.0 Flash Experimentalaider, thinking· Aider LLM Leaderboards18.2%
122Gemini 2.0 Flash Experimentalthinking· Epoch AI Benchmarking Hub data18.2%
123DeepSeek V2.5· Epoch AI Benchmarking Hub data17.8%
124GPT OSS 20bofficial model card Table 3 · reasoning low · published setting; Table 3 does not further specify tool access · pass@1· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.16.6%
125Qwen 2.5 Coder 32B Instruct· Epoch AI Benchmarking Hub data16.4%
126Llama 4 Maverick· Epoch AI Benchmarking Hub data15.6%
127Llama 4 Maverickaider· Aider LLM Leaderboards15.6%
128Yi Lightning· Epoch AI Benchmarking Hub data12.9%
129Yi Lightningaider· Aider LLM Leaderboards12.9%
130C4AI Command A 03 2025· Epoch AI Benchmarking Hub data12%
131Codestral 25.012501· Epoch AI Benchmarking Hub data11.1%
132openhands-lm-32b-v0.1· Epoch AI Benchmarking Hub data10.2%
133openhands-lm-32b-v0.1aider· Aider LLM Leaderboards10.2%
134GPT-4.1 nano· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim
Values from 2 sources
9.8%
135GPT-4.1 nanoaider· Aider LLM Leaderboards8.9%
136Qwen 2.5 Coder 32B Instructaider· Aider LLM Leaderboards8%
137Gemma 3 27B IT· Epoch AI Benchmarking Hub data4.9%
138Gemma 3 27B ITaider· Aider LLM Leaderboards4.9%
139GPT-4o mini· Epoch AI Benchmarking Hub data3.6%
140GPT-4o miniaider· Aider LLM Leaderboards3.6%

Showing 101–140 of 140 results.

Data credits: Epoch AI Benchmarking Hub, CC BY 4.0 (licence) · Aider LLM Leaderboards (Aider-AI), Apache-2.0 (licence) · Steel.dev Agent Leaderboards, MIT (licence) · BenchBoard by SYuan03 (github.com/SYuan03/benchboard). MIT covers its code and data organization; no results licence is inferred. Numeric relays retain the original stating-source links. · Data by LLM Stats