Skip to content

Benchmark

Natural2Code

Scores are percentages.

Models measured
8
Scores recorded
8
Sources
1
IQ weight
1.89

Score distribution

78.1%
Mean
80.7%
Median
10.5%
Std dev
70.3%
10th percentile
85.4%
90th percentile
56%
Min
56%92.9%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.89 for chat models, measured over 8 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Gemini 2.0 Flash· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim92.9%
2Gemini 1.5 Pro· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim85.4%
3Gemma 3 27B IT· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim84.5%
4Gemma 3 12B IT· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim80.7%
5Gemini 1.5 Flash· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim79.8%
6Gemini 1.5 Flash-8B· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim75.5%
7Gemma 3 4B IT· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim70.3%
8Gemma 3 1b it· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim56%

Data credits: Data by LLM Stats