Skip to content

Benchmark

Aider Polyglot

Coding benchmarks compare models on programming tasks. Scores are percentages. Higher is better.

Models measured
63
Scores recorded
140
Sources
5
IQ weight
2.11

Score distribution

52.7%
Mean
56.4%
Median
24.3%
Std dev
16.4%
10th percentile
81.3%
90th percentile
3.6%
Min
3.6%89.4%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 2.11 for chat models, measured over 63 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Claude Opus 4.5launch-page coding chart· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.89.4%
2GPT 5· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim88%
3GPT 5aider-reasoning-high, high· Aider LLM Leaderboards88%
4GPT 5high· Epoch AI Benchmarking Hub data
Values from 2 sources
88%
5GPT 5aider-reasoning-medium, medium· Aider LLM Leaderboards86.7%
6GPT 5medium· Epoch AI Benchmarking Hub data86.7%
7O3 Proaider-reasoning-high, high· Aider LLM Leaderboards84.9%
8O3 Prohigh· Epoch AI Benchmarking Hub data
Values from 2 sources
84.9%
9Gemini 2.5 Pro32k· Epoch AI Benchmarking Hub data83.1%
10Gemini 2.5 Pro32k, think· Steel.dev Agent Leaderboards data83.1%
11Gemini 2.5 Proaider-reasoning-32k, 32k think· Aider LLM Leaderboards83.1%
12Gemini 2.5 Pro· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim
Values from 2 sources
82.2%
13GPT 5aider-reasoning-low, low· Aider LLM Leaderboards81.3%
14GPT 5low· Epoch AI Benchmarking Hub data
Values from 2 sources
81.3%
15OpenAI o3· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim
Values from 3 sources
81.3%
16OpenAI o3aider-reasoning-high, high· Aider LLM Leaderboards81.3%
17OpenAI o3high· Epoch AI Benchmarking Hub data
Values from 2 sources
81.3%
18Grok 4· Epoch AI Benchmarking Hub data79.6%
19Grok 4aider-reasoning-high, high· Aider LLM Leaderboards79.6%
20Grok 4high· Steel.dev Agent Leaderboards data79.6%
21Gemini 2.5 Proaider-reasoning-default, default think· Aider LLM Leaderboards79.1%
22Gemini 2.5 Prothink· Steel.dev Agent Leaderboards data79.1%
23Claude Sonnet 4.5launch-page coding chart· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.78.8%
24Gemini 2.5 Pro Preview 05-06· Epoch AI Benchmarking Hub data
Values from 2 sources
76.9%
25Gemini 2.5 Pro Preview 05-06aider· Aider LLM Leaderboards76.9%
26OpenAI o3aider· Aider LLM Leaderboards76.9%
27OpenAI o3medium· Epoch AI Benchmarking Hub data76.9%
28DeepSeek V3.2 Exp· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim
Values from 2 sources
74.5%
29DeepSeek Reasoner· Epoch AI Benchmarking Hub data74.2%
30DeepSeek V3.2 Expthinking· Epoch AI Benchmarking Hub data74.2%
31Gemini 2.5 Pro· Epoch AI Benchmarking Hub data72.9%
32Gemini 2.5 Pro Preview 03-25· Epoch AI Benchmarking Hub data
Values from 2 sources
72.9%
33Gemini 2.5 Pro Preview 03-25aider· Aider LLM Leaderboards72.9%
34Claude Opus 432k· Epoch AI Benchmarking Hub data72%
35Claude Opus 432k, thinking· Steel.dev Agent Leaderboards data72%
36Claude Opus 4aider-reasoning-32k, 32k thinking· Aider LLM Leaderboards72%
37o4 Mini· Steel.dev Agent Leaderboards data
Values from 2 sources
72%
38o4 Miniaider-reasoning-high, high· Aider LLM Leaderboards72%
39o4 Minihigh· Epoch AI Benchmarking Hub data72%
40DeepSeek R1· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim
Values from 2 sources
71.6%
41Claude Opus 4· Epoch AI Benchmarking Hub data70.7%
42Claude Opus 4aider-reasoning-off, no think· Aider LLM Leaderboards70.7%
43DeepSeek Chat· Epoch AI Benchmarking Hub data70.2%
44DeepSeek V3.2 Expaider, Chat· Aider LLM Leaderboards70.2%
45DeepSeek V3.1· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim68.4%
46o3 Mini· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim66.7%
47Claude Sonnet 3.732k· Epoch AI Benchmarking Hub data64.9%
48Claude Sonnet 3.7aider-reasoning-32k, 32k thinking tokens· Aider LLM Leaderboards64.9%
49Gemini 2.5 Flash· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim61.9%
50Qwen3 Coder· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim61.8%

Showing 1–50 of 140 results.

Data credits: Epoch AI Benchmarking Hub, CC BY 4.0 (licence) · Aider LLM Leaderboards (Aider-AI), Apache-2.0 (licence) · Steel.dev Agent Leaderboards, MIT (licence) · BenchBoard by SYuan03 (github.com/SYuan03/benchboard). MIT covers its code and data organization; no results licence is inferred. Numeric relays retain the original stating-source links. · Data by LLM Stats