Skip to content

Benchmark

OfficeQA Pro

Scores are percentages. Higher is better.

Models measured
22
Scores recorded
34
Sources
2
IQ weight
1.14

Score distribution

60.7%
Mean
63.3%
Median
11%
Std dev
47.1%
10th percentile
71.1%
90th percentile
18.1%
Min
18.1%80.6%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.14 for chat models, measured over 22 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Claude Opus 4.7exact-match grading · 0% allowable relative error · system-card table· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.80.6%
2Claude Fable 5.2· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim72.9%
3Doubao Seed 2.1 Pro· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim72.2%
4Doubao Seed 2.1 Turbo· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim71.1%
5Claude Fable 5.1Opus 5.5 System Card capability section · same Opus 5.5 card setup· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.69%
6ERNIE 5.5 Flash Preview· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim67.9%
7Claude Opus 5.5· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim67.7%
8Claude Opus 5.5Opus 5.5 System Card capability section · 133-question subset · extracted-text sandbox + code · max · 5 runs· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.67.7%
9Kimi K3.1 Pro· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim67.4%
10Claude Opus 5Opus 5.5 System Card capability section · same Opus 5.5 card setup· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.66.9%
11Claude Opus 4.8· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim66.2%
12Hy4 preview· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim66.2%
13Hy4 previewofficial model-card appendix· Reported by Tencent Hy· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.66.2%
14Claude Sonnet 5.5· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim65.6%
15Claude Sonnet 5.5adaptive thinking · max effort · default sampling · 5-trial mean · context ≤1M unless noted · 133-question subset · preselected extracted text + code · max · 5 runs· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.65.6%
16Claude Opus 4.7launch-page office-tasks chart · published value conflicts with 80.6 in system card· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.65%
17Kimi K3· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim63.3%
18Kimi K3max effort · temp 1.0 · top_p 1.0· Reported by Moonshot AI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.63.3%
19GLM 5.3 Flash· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim62.4%
20GLM 5.3 FlashTreasury Bulletin / 无嵌入文本· Reported by Z.ai· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.62.4%
21Claude Haiku 5.5· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim60.3%
22Claude Haiku 5.5adaptive thinking · max effort · default sampling · 5-trial mean · context ≤1M unless noted · 133-question subset · preselected extracted text + code · max · 5 runs· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.60.3%
23Claude Sonnet 5· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim59.4%
24Claude Sonnet 5133-question subset · extracted-text sandbox + code · max effort · 5 trials· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.59.4%
25Claude Fable 5Fable/Mythos 5 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.57.9%
26GPT 5.5· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim54.1%
27GPT 5.5xhigh· Reported by OpenAI· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.54.1%
28Claude Sonnet 4.6extracted-text sandbox + code · max effort · Sonnet 5 card rerun· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.53.4%
29GPT 5.5Fable/Mythos 5 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.52.6%
30Claude Opus 4.8Fable/Mythos 5 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.48.1%
31Claude Haiku 4.5adaptive thinking · max effort · default sampling · 5-trial mean · context ≤1M unless noted · 133-question subset · preselected extracted text + code · fixed thinking budget · 5 runs· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.47.1%
32MiniMax M3· LLM Stats data· Reported by the model's developer. The source relaying it did not verify it. claim45.1%
33MiniMax M3Claude Code · filesystem delivery · exact match· Reported by MiniMax Research· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.45.1%
34Gemini 3.1 Pro PreviewFable/Mythos 5 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted· Reported by Anthropic· via BenchBoard docs· Reported by the model's developer. The source relaying it did not verify it.18.1%

Data credits: BenchBoard by SYuan03 (github.com/SYuan03/benchboard). MIT covers its code and data organization; no results licence is inferred. Numeric relays retain the original stating-source links. · Data by LLM Stats