Skip to content

Benchmark

Bird-SQL (dev)

Scores are percentages.

Models measured
9
Scores recorded
9
Sources
1
IQ weight
2.02

Score distribution

42.7%
Mean
41.9%
Median
14.7%
Std dev
36.3%
10th percentile
56.9%
90th percentile
6.4%
Min
6.4%57.4%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 2.02 for chat models, measured over 9 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Gemini 2.0 Flash Lite· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim57.4%
2Gemini 2.0 Flash· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim56.9%
3Gemma 3 27B IT· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim54.4%
4Gemma 3 12B IT· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim47.9%
5Granite 4.2 30B· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim41.9%
6Nemotron 3 Super 120B A12B· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim41.8%
7Granite 4.2 8B· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim41.1%
8Gemma 3 4B IT· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim36.3%
9Gemma 3 1b it· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim6.4%

Data credits: Data by LLM Stats