Skip to content

Benchmark

MT-Bench

People compare model answers and choose which they prefer. Scores are percentages.

Models measured
9
Scores recorded
9
Sources
1
IQ weight
0.56

Score distribution

78.1%
Mean
86.3%
Median
24.9%
Std dev
76.8%
10th percentile
91.7%
90th percentile
9%
Min
9%93.5%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.56 for chat models, measured over 9 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Qwen2.5 72B Instruct· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim0.935%
2Llama 3.3 Nemotron Super 49B v1· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim0.917%
3DeepSeek V2.5· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim0.902%
4Qwen2.5 7B Instruct· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim0.875%
5Mistral Large 2· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim0.863%
6Qwen2 7B Instruct· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim0.841%
7Mistral Small 24B Instruct2501· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim0.835%
8Pixtral 12B2409· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim0.768%
9Llama 3.1 Nemotron 70B Instruct· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim0.0899%

Data credits: Data by LLM Stats