Skip to content

Benchmark

MuirBench

Scores are percentages.

Models measured
10
Scores recorded
12
Sources
1
IQ weight
1.64

Score distribution

73.4%
Mean
77.5%
Median
7%
Std dev
62.9%
10th percentile
80.1%
90th percentile
59.2%
Min
59.2%80.3%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.64 for chat models, measured over 9 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Qwen3 VL 32B Thinking· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it.80.3%
2Qwen3 VL 235Bthinking· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it.80.1%
3Doubao Seed 1.8· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim78.7%
4Seed 2.0 Mini· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim78%
5qwen3-vl-30bthinking· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it.77.6%
6Inkling 2 Mini· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim77.5%
7Qwen: Qwen3 VL 8B Thinking· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it.76.8%
8Qwen3 VL 235B· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it.72.8%
9Qwen3 VL 32B Instruct· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it.72.8%
10Qwen3 VL 8B Instruct· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it.64.4%
11qwen3-vl-30b· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it.62.9%
12Qwen2.5-Omni 7B· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim59.2%

Data credits: Data by LLM Stats