Skip to content

Benchmark

Aider Polyglot

Coding benchmarks compare models on programming tasks. Scores are percentages. Higher is better.

Models measured
63
Scores recorded
140
Sources
5
IQ weight
2.11

Score distribution

52.7%
Mean
56.4%
Median
24.3%
Std dev
16.4%
10th percentile
81.3%
90th percentile
3.6%
Min
3.6%89.4%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 2.11 for chat models, measured over 63 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
51o1aider-reasoning-high, high· Aider LLM Leaderboards61.7%
52o1high· Epoch AI Benchmarking Hub data61.7%
53Claude Sonnet 432k· Epoch AI Benchmarking Hub data61.3%
54Claude Sonnet 4aider-reasoning-32k, 32k thinking· Aider LLM Leaderboards61.3%
55Claude Sonnet 3.7· Epoch AI Benchmarking Hub data60.4%
56Claude Sonnet 3.7aider-reasoning-off, no thinking· Aider LLM Leaderboards60.4%
57o3 Miniaider-reasoning-high, high· Aider LLM Leaderboards60.4%
58o3 Minihigh· Epoch AI Benchmarking Hub data60.4%
59Kimi K2 Instruct· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim
Values from 2 sources
60%
60Qwen 3 235B A22B· Epoch AI Benchmarking Hub data59.6%
61Qwen3 235B A22B Instruct 2507· Epoch AI Benchmarking Hub data
Values from 2 sources
59.6%
62Kimi K2aider· Aider LLM Leaderboards59.1%
63Kimi K2 0905· Epoch AI Benchmarking Hub data59.1%
64DeepSeek R1aider· Aider LLM Leaderboards56.9%
65Claude Sonnet 4· Epoch AI Benchmarking Hub data56.4%
66Claude Sonnet 4aider-reasoning-off, no thinking· Aider LLM Leaderboards56.4%
67DeepSeek V3· Epoch AI Benchmarking Hub data
Values from 2 sources
55.1%
68DeepSeek V3aider· Aider LLM Leaderboards55.1%
69Gemini 2.5 Flash Preview 05-20 (no think)23k· Epoch AI Benchmarking Hub data55.1%
70Gemini 2.5 Flash Preview 05-20 (no think)aider-reasoning-24k, 24k think· Aider LLM Leaderboards55.1%
71o3 Miniaider-reasoning-medium, medium· Aider LLM Leaderboards53.8%
72o3 Minimedium· Epoch AI Benchmarking Hub data53.8%
73Grok 3aider· Aider LLM Leaderboards53.3%
74Grok 3beta· Epoch AI Benchmarking Hub data53.3%
75GPT 4.1· Epoch AI Benchmarking Hub data
Values from 2 sources
52.4%
76GPT 4.1aider· Aider LLM Leaderboards52.4%
77Claude 3.5 Sonnet· Epoch AI Benchmarking Hub data51.6%
78Claude 3.5 Sonnetaider· Aider LLM Leaderboards51.6%
79Qwen3 Next 80B A3B Instruct· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim49.8%
80Grok 3 Miniaider-reasoning-high, high· Aider LLM Leaderboards49.3%
81Grok 3 Minihigh, beta· Epoch AI Benchmarking Hub data49.3%
82DeepSeek V3aider, prev· Aider LLM Leaderboards48.4%
83gemini-2.5-flash-preview-04-17· Epoch AI Benchmarking Hub data47.1%
84gemini-2.5-flash-preview-04-17aider, default· Aider LLM Leaderboards47.1%
85Magistral Medium· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim47.1%
86GPT-4.5 Preview· Epoch AI Benchmarking Hub data44.9%
87GPT-4.5 Previewaider· Aider LLM Leaderboards44.9%
88GPT OSS 120Bofficial model card Table 3 · reasoning high · published setting; Table 3 does not further specify tool access · pass@1· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.44.4%
89Gemini 2.5 Flash Preview 05-20 (no think)· Epoch AI Benchmarking Hub data44%
90Gemini 2.5 Flash Preview 05-20 (no think)aider-reasoning-off, no think· Aider LLM Leaderboards44%
91GPT OSS 120Baider-reasoning-high, high· Aider LLM Leaderboards41.8%
92GPT OSS 120Bhigh· Epoch AI Benchmarking Hub data41.8%
93Qwen3 32B· Epoch AI Benchmarking Hub data40%
94Qwen3 32Baider· Aider LLM Leaderboards40%
95gemini-exp· Epoch AI Benchmarking Hub data38.2%
96gemini-expaider· Aider LLM Leaderboards38.2%
97gemini-2.0-pro-exp-02-05· Epoch AI Benchmarking Hub data35.6%
98gemini-2.0-pro-exp-02-05aider· Aider LLM Leaderboards35.6%
99GPT-4.1 mini· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim
Values from 2 sources
34.7%
100Grok 3 Miniaider-reasoning-low, low· Aider LLM Leaderboards34.7%

Showing 51–100 of 140 results.

Data credits: Epoch AI Benchmarking Hub, CC BY 4.0 (licence) · Aider LLM Leaderboards (Aider-AI), Apache-2.0 (licence) · Steel.dev Agent Leaderboards, MIT (licence) · BenchBoard by SYuan03 (github.com/SYuan03/benchboard). MIT covers its code and data organization; no results licence is inferred. Numeric relays retain the original stating-source links. · Data by LLM Stats