Skip to content

Anthropic · Chat

Claude Opus 4.8

Anthropic 🇺🇸ChatClaude familyOne of its sources says the weights aren't publicly available.The maker currently lists 1 Sep 2027 as the deprecation date. Plans can change.

Newer version: Claude Opus 5.5. We compared release dates and version names in our catalogue.

From Neosantara's listing:
Claude Opus 4.8 is Anthropic's model for complex reasoning and advanced analysis.
llminfo IQ
139.7
Maker's price
$10.00 blended / 1M
Max context
1M
Listings
119

Evidence and sources

llminfo IQ 139.7, ranked among chat models. From 478 benchmarks. Plenty of data behind it.How llminfo IQ is computedCoding IQ 120.8, 95% interval 116–125. From 35 benchmark families across 16 sources. Plenty of data behind it. Uses the model's best published results. It has its own scale, separate from llminfo IQ.How Coding IQ is computedGood value (52.9). Cheaper per task than its benchmark results predict. From 70 benchmarks with measured task costs. Plenty of data behind it.How Value for money worksYap score: tokens used per coding task vs. models with the same resultsTerse (40 ±10). Generates fewer tokens than models with similar coding results. Measured at default effort. A fair amount of data behind it. By effort: default 40 · low 47 · medium 52 · high 46 · max 67.Median 16,437 output tokens per task at default effort.How the Yap score worksReleased 28 May 2026Maker HQ: US (according to Anthropic's own website)
All rankings: #78 on OpenRouter today #79 on OpenRouter this month #89 on OpenRouter this week #197 on NanoGPT recentlySources: Anthropic API data, Heroku Managed Inference data, LiteLLM data, Models.dev data, Neosantara Inference data, llminfo researchPrimary data from Anthropic API from Heroku Managed Inference: primary mode from LiteLLM: deprecation date from Models.dev: weights availability from Neosantara Inference: description from llminfo research: release date
Tags from its sources:instructtoolsvisionreasoningdocumentproprietarycodingagenticlong-context

Advantages and trade-offs

Compared with 10 chat models, picked mainly by popularity and IQ

Advantages

Trade-offs

No open weights

The weights aren't public, so you can't run it yourself.

We only make a claim when there's enough data on models of the same type. Updated 22 Nov, 04:34 UTC.Prices come from the model's maker where they sell it, otherwise their official partner, otherwise the market median. This comparison uses 7 maker's prices; 3 market median prices.

Input & output

as stated by the model's providers
Text
Images
Files
Text

Capabilities

Supported operations and provider-stated features

VisionReasoningFunction callingStructured outputPrompt cachingPDF inputComputer useWeb searchBatch API

Listings

 Anthropicclaude-opus-4-8Sources: Anthropic API data, GenAI Prices data, LiteLLM data, Models.dev data1M$5.00cache read $0.50batch $2.50$25.00cache write 5m $6.25cache write 1h $10.00batch $12.50—
 Jzstokenclaude-opus-4-8Source: Models.dev data1M$1.17cache read $0.1168$5.84cache write $1.46—
 Xpersonaclaude-opus-4-8Sources: Models.dev data, Xpersona data200K$1.50cache read $0.15$9.25$0.001
 Tokensmindclaude-opus-4-8Source: TokensMindNot published by the provider$1.75$8.75—
 Xxapiclaude-opus-4-8Source: Models.dev data1M$1.923cache read $0.1923$9.615cache write $2.404—
 OpenRouteranthropic/claude-opus-4.8:batch · batchSource: OpenRouter data docs1M$2.50cache read $0.25$12.50cache write 5m $3.125cache write 1h $5.00—
 Vercelanthropic/claude-opus-4.8-batch · batchSource: Vercel AI Gateway data1M$2.50cache read $0.25$12.50—
 Korouterclaude-opus-4-8Source: Models.dev data1M$3.25cache read $0.325$16.25cache write $4.06—
 Kunavoclaude-opus-4-8Source: Models.dev data1M$3.50cache read $0.35$17.50cache write $4.375—
 Meganovaanthropic/claude-opus-4-8Source: MegaNova data1M$4.25$21.25—

12 listings without a price are hidden.

Benchmarks

Named effort levels run from lowest to highest. Select a result to see its source details.

Human preference

Arena ELO66 related benchmarks · 128 results
Source: LMArena data · Nov 2026
Arena ELO
Chinese
Creative Writing
Document
Document Style-Controlled
English
Show 116 more resultsHide 116 results
Exclude Ties
Expert
French
German
Hard Prompts
Hard Prompts English
Industry Mathematical
Industry Medicine and Healthcare
Industry Software and It Services
Japanese
Korean
Longer Query
Multi Turn
Non English
Polish
Russian
Search Factuality
Spanish
Style-Controlled
Text Factuality
Text Factuality - Chinese
Text Factuality - Creative Writing
Text Factuality - English
Text Factuality - Exclude Ties
Text Factuality - Expert
Text Factuality - French
Text Factuality - German
Text Factuality - Hard Prompts
Text Factuality - Hard Prompts English
Text Factuality - Industry Mathematical
Text Factuality - Japanese
Text Factuality - Korean
Text Factuality - Longer Query
Text Factuality - Multi Turn
Text Factuality - Non English
Text Factuality - Polish
Text Factuality - Russian
Text Factuality - Spanish
Vision
Vision - Chinese
Vision - Creative Writing Vision
Vision - Diagram
Vision - English
Vision - Homework
Vision - Humor
Vision Style-Controlled
Design Arena EloSource: Design Arena Leaderboards data · Nov 2026

Agentic

Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI GPT-5.6 launch table
Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Qwen表:Overall Score
BrowseComp5 results
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Opus 5 System Card capability summary · adaptive thinking · max effort unless noted
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI GPT-5.6 launch table
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
Design Arena EloSource: Design Arena Leaderboards data · Nov 2026
Source: LLM Stats data
Setting not stated
Reported by Meta, Meta Superintelligence Labs · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
max · Vals AI result
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Opus 4.8 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted
GAIA4 results
Source: GAIA Leaderboard data · Jul 2026
Difficulty level
Difficulty not reported
GDPval-AA v26 results
Reported by Z.ai · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Artificial Analysis / Z.ai转载
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Fable/Mythos 5 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted
Opus 4.8 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted
Opus 5 System Card capability summary · adaptive thinking · max effort unless noted
Reported by Meta · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
max · Artificial Analysis leaderboard
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI GPT-5.6 launch table
JobBench2 results
Reported by Meta, Meta Superintelligence Labs · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
max · OpenCode · Grok 4.3 judge · official JobBench results
Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
MCP Atlas3 results
Reported by Meta, Meta Superintelligence Labs · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
max · Scale AI MCP Atlas
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Opus 4.8 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
MCPMark VerifiedSource: MCPMark Verified leaderboard docs
OpenHandsSource: OpenHands Index data
OSWorld6 related benchmarks · 12 results
OSWorld
Source: Steel.dev Agent Leaderboards data · May 2026
2.0 · Binary
Reported by Meta · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
max · 108 tasks · GUI-only · no exec · 500 steps · binary completion
2.0 · Partial
Reported by Meta · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
max · 108 tasks · GUI-only · no exec · 500 steps · partial reward
OSWorld 2
Source: Epoch AI Benchmarking Hub data
Source: Steel.dev Agent Leaderboards data · Jun 2026
Setting not stated
OSWorld 2.0
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Opus 5 System Card setting
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OSWorld 2.0 · OpenAI GPT-5.6 launch table
Verified
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Fable/Mythos 5 System Card setting
Opus 4.8 System Card setting
Reported by Meta, Meta Superintelligence Labs · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
max · 361-task split excluding Google Drive · GUI-only · 1920×1080 · 200 steps · pass@1
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
SWE-Bench Pro8 results
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Fable/Mythos 5 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted
Opus 4.8 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted
Opus 5 System Card capability summary · adaptive thinking · max effort unless noted
Reported by Meta, Meta Superintelligence Labs · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
max · Scale AI · mini-swe-agent for Muse Spark 1.1
Reported by SpaceXAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI GPT-5.6 launch table
Show 2 more resultsHide 2 results
Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Qwen表:Claude Code · temp 1.0 · top_p 0.95 · 256K · refined tasks
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
Source: OpenRouter benchmark exports
28 May 2026 version · 90-day results
Across 5 providers lowest 76.1% median 76.6% highest 76.9%OpenRouter's automatic routing 76.6%
Show results from 5 providers and OpenRouter's automatic routingHide results from 5 providers and OpenRouter's automatic routing
  • Claude Platform on AWS
  • Anthropic
  • Azure
  • OpenRouter's automatic routing
  • Google
  • Amazon Bedrock
Terminal-Bench 2.05 related benchmarks · 12 results
Terminal-Bench 2.0
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Terminal-Bench 2.1
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Fable/Mythos 5 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted
Opus 4.8 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted
Reported by Meta, Meta Superintelligence Labs · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
max · 89 tasks · bash-only · pass@1 · five-attempt mean
Reported by SpaceXAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI GPT-5.6 launch table
Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Qwen表:Qwen 采用 Claude Code avg@10、5h、max_tokens 131072;友商取公开最高分
Reported by Z.ai · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Terminal-Bench 3.0
Reported by Z.ai · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Terminal-Bench 4.0
Reported by Terminal-Bench / Stanford / Harbor / Laude Institute · via BenchBoard docs
Terminal-Bench 4.0 official leaderboard · Claude Code · max effort
Source: LLM Stats data
Setting not stated
WebArena-Verifiedmax · OpenClaw · DOM + screenshots · full 812 tasks · 150 tool calls · pass@1 · Reported by Meta · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.

Coding

BenchCAD2 results
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Python tool enabled
CyberGymSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
ExploitBenchOpenAI GPT-5.6 launch table · Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
FrontierBench v0.1Opus 5 System Card capability summary · adaptive thinking · max effort unless noted · Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
FrontierCode4 related benchmarks · 7 results
FrontierCode 1.0
Source: FrontierCode data
1.1 Main
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Opus 5 System Card capability summary · adaptive thinking · max effort unless noted
Diamond
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Fable/Mythos 5 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted · Diamond subset
FrontierCode 1.1
Source: FrontierCode data
Source: LLM Stats data
Setting not stated
FrontierSWE2 results
Source: LLM Stats data
Setting not stated
Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Qwen表:Claude Code;对照为 2026-08-03 官方榜 MEAN@5
MLS-Bench-LiteQwen表:Claude Code · 5h · max_tokens 131072;对照来自官方榜 · Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
PaperBenchQwen表:BasicAgent · Code-Dev · Claude Opus 4.6 judge · 3次均值 · 最长12h · Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
QwenSVGBenchQwen内部双语 SVG 基准 · 自动渲染 + 多模态评判 · BT/Elo · Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
QwenSWEBenchQwen内部基准:Claude Code · avg@3 · 8h · 256K · Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
SWE-Marathon7 related benchmarks · 13 results
SWE-Marathon
Reported by SpaceXAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
SWE-bench Multilingual
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Opus 4.8 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted
Opus 5 System Card capability summary · adaptive thinking · max effort unless noted
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
SWE-bench Multimodal
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Opus 4.8 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted
Opus 5 System Card capability summary · adaptive thinking · max effort unless noted
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
SWE-bench Verified
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Fable/Mythos 5 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted
Opus 4.8 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
SWE-bench-Live Lite
Source: SWE-bench-Live leaderboard data · Sep 2026
scaffold: aiwork.code, set: lite, instances: 300, effort: xhigh
SWE-rebench 2026-03
Source: SWE-rebench
Show 1 more resultHide 1 result
SWE-rebench 2026-04
Source: SWE-rebench
ValsSource: Vals AI

Reasoning

ARC-AGI4 related benchmarks · 10 results
ARC-AGI
Source: Epoch AI Benchmarking Hub data
Reasoning effort
ARC-AGI 2
Source: Epoch AI Benchmarking Hub data
Reasoning effort
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Opus 5 System Card capability summary · adaptive thinking · max effort unless noted
ARC-AGI-1
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Opus 5 System Card capability summary · adaptive thinking · max effort unless noted
ARC-AGI-3
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI GPT-5.6 launch table
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Fable/Mythos 5 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted
Opus 4.8 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted
Opus 5 System Card capability summary · adaptive thinking · max effort unless noted
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI GPT-5.6 launch table
Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Qwen表:600题公开子集 · Pass@1
Reported by Z.ai · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Big Finance BenchOpenAI GPT-5.6 launch table · Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
BioMysteryBenchFable/Mythos 5 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted · Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Blueprint-Bench 2Fable/Mythos 5 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted · Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
CoWorkBenchReported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
DeepSearchQA2 results
Reported by Meta · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
max · browser search/open/find · gpt-oss-120b answer extraction
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
GDP.pdf2 results
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Fable/Mythos 5 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI GPT-5.6 launch table
GeneBench ProOpenAI GPT-5.6 launch table · Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Fable/Mythos 5 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted · no tools
Opus 4.8 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted · no tools
Opus 5 System Card capability summary · adaptive thinking · max effort unless noted · no tools
Reported by Meta · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
max · no tools · full 2,500 questions · o3-mini judge
Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Fable/Mythos 5 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted · with tools
Opus 4.8 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted · with tools
Opus 5 System Card capability summary · adaptive thinking · max effort unless noted · with tools
Reported by Meta, Meta Superintelligence Labs · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
max · full 2,500 questions · bash + browser tools · o3-mini judge
Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Reported by Z.ai · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
LifeSciBenchOpenAI GPT-5.6 launch table · Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OfficeQA Pro2 results
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Fable/Mythos 5 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
internal pinned harness · adaptive thinking · max · 108 tasks · 3 trials · Pass@1
Reported by Meta, Meta Superintelligence Labs · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
max · Toolathlon-Verified · official harness
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI GPT-5.6 launch table
Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Qwen表:Verified · Pass@1
Reported by Z.ai · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
WideSearchQwen表:友商=Claude Code,Qwen=Qwen-Agent · 4次平均 item-F1 · Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.

General

Science

CritPT2 results
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Fable/Mythos 5 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted
GPQASource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
GPQA Diamond12 results
Source: Epoch AI Benchmarking Hub data
Reasoning effort
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI GPT-5.6 launch table
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Opus 4.8 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted
Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Source: OpenRouter benchmark exports
28 May 2026 version · 90-day results
Across 5 providers lowest 89.8% median 90.1% highest 90.1%OpenRouter's automatic routing 90%
Show results from 5 providers and OpenRouter's automatic routingHide results from 5 providers and OpenRouter's automatic routing
  • Claude Platform on AWS
  • Google
  • Azure
  • Amazon Bedrock
  • OpenRouter's automatic routing
  • Anthropic

Math

ArxivMathFable/Mythos 5 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted · Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
FrontierMath5 related benchmarks · 6 results
Tier 1-3 (v2)
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Tier 4 (v2)
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
v2 · GPT-5.6 launch snapshot · 2026-07-09

Vision

ChartQAProOpus 4.8 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted · Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Fable/Mythos 5 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted · reasoning
Reported by Meta, Meta Superintelligence Labs · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
max · with code execution · 1,000 validation questions · gpt-oss-120b high judge
CharXiv-RSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Opus 4.8 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated

Long context

LongBench v2Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.

Multilingual

EKI EstonianSource: Keelemudelite mõõdupuu data docs

Instruction following

IFBenchReported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.

Knowledge

IncludeSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim

OCR

Factuality

Speech

Other

AndroidBenchQwen表:95题公开子集 · avg@3 · Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Artificial Analysis BriefcaseOpus 5 System Card capability summary · adaptive thinking · max effort unless noted · Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
BlenderWorld15 related benchmarks · 15 results
Art & Animation
Reported by BlenderWorld authors / Alibaba Token Hub · via BenchBoard docs
benchmark-owner paper run · 30 category tasks · Blender 4.2.10 / Blender MCP paper setup · GPT-5.6 Terra main judge · generation-agent harness and reasoning effort not stated on public leaderboard
Architecture & Interiors
Reported by BlenderWorld authors / Alibaba Token Hub · via BenchBoard docs
benchmark-owner paper run · 72 category tasks · Blender 4.2.10 / Blender MCP paper setup · GPT-5.6 Terra main judge · generation-agent harness and reasoning effort not stated on public leaderboard
Characters & Animals
Reported by BlenderWorld authors / Alibaba Token Hub · via BenchBoard docs
benchmark-owner paper run · 25 category tasks · Blender 4.2.10 / Blender MCP paper setup · GPT-5.6 Terra main judge · generation-agent harness and reasoning effort not stated on public leaderboard
Fully satisfied rubric
Reported by BlenderWorld authors / Alibaba Token Hub · via BenchBoard docs
benchmark-owner paper run · 223 tasks · Blender 4.2.10 / Blender MCP paper setup · GPT-5.6 Terra main judge · generation-agent harness and reasoning effort not stated on public leaderboard
Image/video-input overall score
Reported by BlenderWorld authors / Alibaba Token Hub · via BenchBoard docs
benchmark-owner paper run · 145 image or video reference tasks · Blender 4.2.10 / Blender MCP paper setup · GPT-5.6 Terra main judge · generation-agent harness and reasoning effort not stated on public leaderboard
Nature & Landscapes
Reported by BlenderWorld authors / Alibaba Token Hub · via BenchBoard docs
benchmark-owner paper run · 21 category tasks · Blender 4.2.10 / Blender MCP paper setup · GPT-5.6 Terra main judge · generation-agent harness and reasoning effort not stated on public leaderboard
Show 9 more resultsHide 9 results
Overall score
Reported by BlenderWorld authors / Alibaba Token Hub · via BenchBoard docs
benchmark-owner paper run · 223 tasks · Blender 4.2.10 / Blender MCP paper setup · GPT-5.6 Terra main judge · generation-agent harness and reasoning effort not stated on public leaderboard
Procedural & Game Assets
Reported by BlenderWorld authors / Alibaba Token Hub · via BenchBoard docs
benchmark-owner paper run · 10 category tasks · Blender 4.2.10 / Blender MCP paper setup · GPT-5.6 Terra main judge · generation-agent harness and reasoning effort not stated on public leaderboard
Products & Manufacturing
Reported by BlenderWorld authors / Alibaba Token Hub · via BenchBoard docs
benchmark-owner paper run · 28 category tasks · Blender 4.2.10 / Blender MCP paper setup · GPT-5.6 Terra main judge · generation-agent harness and reasoning effort not stated on public leaderboard
Rendering & Simulation
Reported by BlenderWorld authors / Alibaba Token Hub · via BenchBoard docs
benchmark-owner paper run · 17 category tasks · Blender 4.2.10 / Blender MCP paper setup · GPT-5.6 Terra main judge · generation-agent harness and reasoning effort not stated on public leaderboard
Science & Visualization
Reported by BlenderWorld authors / Alibaba Token Hub · via BenchBoard docs
benchmark-owner paper run · 9 category tasks · Blender 4.2.10 / Blender MCP paper setup · GPT-5.6 Terra main judge · generation-agent harness and reasoning effort not stated on public leaderboard
Text-only overall score
Reported by BlenderWorld authors / Alibaba Token Hub · via BenchBoard docs
benchmark-owner paper run · 78 text-only tasks · Blender 4.2.10 / Blender MCP paper setup · GPT-5.6 Terra main judge · generation-agent harness and reasoning effort not stated on public leaderboard
Vehicles & Machinery
Reported by BlenderWorld authors / Alibaba Token Hub · via BenchBoard docs
benchmark-owner paper run · 11 category tasks · Blender 4.2.10 / Blender MCP paper setup · GPT-5.6 Terra main judge · generation-agent harness and reasoning effort not stated on public leaderboard
Visual-quality dimension score
Reported by BlenderWorld authors / Alibaba Token Hub · via BenchBoard docs
benchmark-owner paper run · 223 tasks · Blender 4.2.10 / Blender MCP paper setup · GPT-5.6 Terra main judge · generation-agent harness and reasoning effort not stated on public leaderboard
Weighted rubric score
Reported by BlenderWorld authors / Alibaba Token Hub · via BenchBoard docs
benchmark-owner paper run · 223 tasks · Blender 4.2.10 / Blender MCP paper setup · GPT-5.6 Terra main judge · generation-agent harness and reasoning effort not stated on public leaderboard
DeepSWE v1.16 results
Reported by Meta, Meta Superintelligence Labs · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
max · mini-swe-agent fork · no internet · 64GB VM · pass@1 · five-attempt mean
Reported by SpaceXAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Reported by Z.ai · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
mini-swe-agent / Z.ai设置
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI GPT-5.6 launch table
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Opus 5 System Card capability summary · adaptive thinking · max effort unless noted
Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Qwen表:Claude Code / mini-SWE-agent 取较高值 · 256K
E-Commerce Bench7 related benchmarks · 7 results
Mean final assets
Reported by E-Commerce Bench authors / Alibaba Qwen · via BenchBoard docs
benchmark-owner 365-day merchant simulation · five independent episodes · up to four stores · initial balance ¥100,000 · max reasoning · exact API snapshot not stated for unversioned labels
AnchorRatio · repeat-order pricing
Reported by E-Commerce Bench authors / Alibaba Qwen · via BenchBoard docs
benchmark-owner 365-day merchant simulation · five independent episodes · up to four stores · initial balance ¥100,000 · max reasoning · exact API snapshot not stated for unversioned labels
BadSpend · fraudulent-supplier spend
Reported by E-Commerce Bench authors / Alibaba Qwen · via BenchBoard docs
benchmark-owner 365-day merchant simulation · five independent episodes · up to four stores · initial balance ¥100,000 · max reasoning · exact API snapshot not stated for unversioned labels
Controllable return
Reported by E-Commerce Bench authors / Alibaba Qwen · via BenchBoard docs
benchmark-owner 365-day merchant simulation · five independent episodes · up to four stores · initial balance ¥100,000 · max reasoning · exact API snapshot not stated for unversioned labels
CSE+ · honest-deal negotiation
Reported by E-Commerce Bench authors / Alibaba Qwen · via BenchBoard docs
benchmark-owner 365-day merchant simulation · five independent episodes · up to four stores · initial balance ¥100,000 · max reasoning · exact API snapshot not stated for unversioned labels
Drawdown / peak assets
Reported by E-Commerce Bench authors / Alibaba Qwen · via BenchBoard docs
benchmark-owner 365-day merchant simulation · five independent episodes · up to four stores · initial balance ¥100,000 · max reasoning · exact API snapshot not stated for unversioned labels
Show 1 more resultHide 1 result
Yuan per tool call
Reported by E-Commerce Bench authors / Alibaba Qwen · via BenchBoard docs
benchmark-owner 365-day merchant simulation · five independent episodes · up to four stores · initial balance ¥100,000 · max reasoning · exact API snapshot not stated for unversioned labels
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
F1
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Fable/Mythos 5 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted · 256K · F1
Opus 4.8 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted · 256K · F1
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Fable/Mythos 5 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted · 256K · accuracy
Opus 4.8 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted · 256K · accuracy
HealthBench2 results
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Fable/Mythos 5 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted
Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Fable/Mythos 5 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted
Opus 5 System Card capability summary · adaptive thinking · max effort unless noted
Reported by Meta · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
max · 525 items · length-normalized · GPT-5.4 low grader
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI GPT-5.6 launch table
Source: Clinical Benchmarks data docs · Reported by the model's developer. The source relaying it did not verify it.
Setting not stated
Setting not stated
Show 1 more resultHide 1 result
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
Horangi36 related benchmarks · 36 results
MRCR v2 · 256KQwen表:MRCR v2 · 256K · 8-needle · Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Nejumi 428 related benchmarks · 28 results
ALT_バイアス
Source: Nejumi Leaderboard 4 data docs
anthropic/claude-opus-4.8: adaptive-thinking-xhigh
ALT_倫理・道徳
Source: Nejumi Leaderboard 4 data docs
anthropic/claude-opus-4.8: adaptive-thinking-xhigh
ALT_制御性
Source: Nejumi Leaderboard 4 data docs
anthropic/claude-opus-4.8: adaptive-thinking-xhigh
ALT_堅牢性
Source: Nejumi Leaderboard 4 data docs
anthropic/claude-opus-4.8: adaptive-thinking-xhigh
ALT_毒性
Source: Nejumi Leaderboard 4 data docs
anthropic/claude-opus-4.8: adaptive-thinking-xhigh
ALT_真実性
Source: Nejumi Leaderboard 4 data docs
anthropic/claude-opus-4.8: adaptive-thinking-xhigh
Show 22 more resultsHide 22 results
AVG_jaster_0shot
Source: Nejumi Leaderboard 4 data docs
anthropic/claude-opus-4.8: adaptive-thinking-xhigh
AVG_jaster_2shots
Source: Nejumi Leaderboard 4 data docs
anthropic/claude-opus-4.8: adaptive-thinking-xhigh
GLP_アプリケーション開発
Source: Nejumi Leaderboard 4 data docs
anthropic/claude-opus-4.8: adaptive-thinking-xhigh
GLP_コーディング
Source: Nejumi Leaderboard 4 data docs
anthropic/claude-opus-4.8: adaptive-thinking-xhigh
GLP_一般的知識
Source: Nejumi Leaderboard 4 data docs
anthropic/claude-opus-4.8: adaptive-thinking-xhigh
GLP_基礎的言語性能
Source: Nejumi Leaderboard 4 data docs
anthropic/claude-opus-4.8: adaptive-thinking-xhigh
GLP_専門的知識
Source: Nejumi Leaderboard 4 data docs
anthropic/claude-opus-4.8: adaptive-thinking-xhigh
GLP_応用的言語性能
Source: Nejumi Leaderboard 4 data docs
anthropic/claude-opus-4.8: adaptive-thinking-xhigh
GLP_情報検索
Source: Nejumi Leaderboard 4 data docs
anthropic/claude-opus-4.8: adaptive-thinking-xhigh
GLP_意味解析
Source: Nejumi Leaderboard 4 data docs
anthropic/claude-opus-4.8: adaptive-thinking-xhigh
GLP_抽象的推論
Source: Nejumi Leaderboard 4 data docs
anthropic/claude-opus-4.8: adaptive-thinking-xhigh
GLP_推論能力
Source: Nejumi Leaderboard 4 data docs
anthropic/claude-opus-4.8: adaptive-thinking-xhigh
GLP_数学的推論
Source: Nejumi Leaderboard 4 data docs
anthropic/claude-opus-4.8: adaptive-thinking-xhigh
GLP_構文解析
Source: Nejumi Leaderboard 4 data docs
anthropic/claude-opus-4.8: adaptive-thinking-xhigh
GLP_知識・質問応答
Source: Nejumi Leaderboard 4 data docs
anthropic/claude-opus-4.8: adaptive-thinking-xhigh
GLP_翻訳
Source: Nejumi Leaderboard 4 data docs
anthropic/claude-opus-4.8: adaptive-thinking-xhigh
GLP_表現
Source: Nejumi Leaderboard 4 data docs
anthropic/claude-opus-4.8: adaptive-thinking-xhigh
GLP_論理的推論
Source: Nejumi Leaderboard 4 data docs
anthropic/claude-opus-4.8: adaptive-thinking-xhigh
GLP_関数呼び出し
Source: Nejumi Leaderboard 4 data docs
anthropic/claude-opus-4.8: adaptive-thinking-xhigh
TOTAL_SCORE
Source: Nejumi Leaderboard 4 data docs
anthropic/claude-opus-4.8: adaptive-thinking-xhigh
アラインメント(ALT)_AVG
Source: Nejumi Leaderboard 4 data docs
anthropic/claude-opus-4.8: adaptive-thinking-xhigh
汎用的言語性能(GLP)_AVG
Source: Nejumi Leaderboard 4 data docs
anthropic/claude-opus-4.8: adaptive-thinking-xhigh
NL2Repo-Bench2 results
Reported by Z.ai · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Setting not stated
Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Qwen表:Claude Code · 禁止访问目标仓库的 Bash 命令
PerceptionBench9 related benchmarks · 9 results
Overall accuracy
Reported by Moonshot AI / PerceptionBench authors · via BenchBoard docs
benchmark-owner evaluation · 3,000 verified questions total · open-ended short answers · unified prompts · max reasoning · main-table result; run count not specified · GPT-oss-120B judge
Attribute accuracy
Reported by Moonshot AI / PerceptionBench authors · via BenchBoard docs
benchmark-owner evaluation · 3,000 verified questions total · open-ended short answers · unified prompts · max reasoning · main-table result; run count not specified · GPT-oss-120B judge
Comparison accuracy
Reported by Moonshot AI / PerceptionBench authors · via BenchBoard docs
benchmark-owner evaluation · 3,000 verified questions total · open-ended short answers · unified prompts · max reasoning · main-table result; run count not specified · GPT-oss-120B judge
Contextual integration accuracy
Reported by Moonshot AI / PerceptionBench authors · via BenchBoard docs
benchmark-owner evaluation · 3,000 verified questions total · open-ended short answers · unified prompts · max reasoning · main-table result; run count not specified · GPT-oss-120B judge
Counting accuracy
Reported by Moonshot AI / PerceptionBench authors · via BenchBoard docs
benchmark-owner evaluation · 3,000 verified questions total · open-ended short answers · unified prompts · max reasoning · main-table result; run count not specified · GPT-oss-120B judge
Depth and 3D perception accuracy
Reported by Moonshot AI / PerceptionBench authors · via BenchBoard docs
benchmark-owner evaluation · 3,000 verified questions total · open-ended short answers · unified prompts · max reasoning · main-table result; run count not specified · GPT-oss-120B judge
Show 3 more resultsHide 3 results
Fine-grained recognition accuracy
Reported by Moonshot AI / PerceptionBench authors · via BenchBoard docs
benchmark-owner evaluation · 3,000 verified questions total · open-ended short answers · unified prompts · max reasoning · main-table result; run count not specified · GPT-oss-120B judge
Localization accuracy
Reported by Moonshot AI / PerceptionBench authors · via BenchBoard docs
benchmark-owner evaluation · 3,000 verified questions total · open-ended short answers · unified prompts · max reasoning · main-table result; run count not specified · GPT-oss-120B judge
Visual relation accuracy
Reported by Moonshot AI / PerceptionBench authors · via BenchBoard docs
benchmark-owner evaluation · 3,000 verified questions total · open-ended short answers · unified prompts · max reasoning · main-table result; run count not specified · GPT-oss-120B judge
Reported by PinchBench · via BenchBoard docs
OpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · average across 5 runs
OpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · fast route · average across 5 runs
Reported by PinchBench · via BenchBoard docs
OpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · best run
OpenClaw · PinchBench v2.0.0 · task count: 147 on live leaderboard / 148 in v2.0.0 release notes · fast route · best run
PLawBenchQwen表:Gemini 3.1 Pro Preview judge · Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
PRBench-FinanceReported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
QwenQoderBenchQwen内部基准:Claude Code · avg@5 · 6h · 256K · Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
QwenReactBenchQwen内部双语 React 基准 · Claude Code harness · 自动渲染 + 多模态评判 · BT/Elo · Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
RecreationBench paper26 related benchmarks · 26 results
Web · Visual assertion score
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 1m context · 128k max output tokens · Web · nominal 50-task split · platform-specific score
Android · Applications with Prog =100%
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 1m context · 128k max output tokens · Android · nominal 50-task split · platform-specific score
Android · Applications with Prog ≥90%
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 1m context · 128k max output tokens · Android · nominal 50-task split · platform-specific score
Android · Programmatic assertion score
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 1m context · 128k max output tokens · Android · nominal 50-task split · platform-specific score
Android · Visual assertion score
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 1m context · 128k max output tokens · Android · nominal 50-task split · platform-specific score
macOS · Applications with Prog =100%
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 1m context · 128k max output tokens · macOS · nominal 50-task split · platform-specific score
Show 20 more resultsHide 20 results
macOS · Applications with Prog ≥90%
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 1m context · 128k max output tokens · macOS · nominal 50-task split · platform-specific score
macOS · Programmatic assertion score
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 1m context · 128k max output tokens · macOS · nominal 50-task split · platform-specific score
macOS · Visual assertion score
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 1m context · 128k max output tokens · macOS · nominal 50-task split · platform-specific score
Overall · Applications with Prog =100%
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 1m context · 128k max output tokens · Overall · 250 held-out tasks · equal-weight platform macro average
Overall · Applications with Prog ≥90%
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 1m context · 128k max output tokens · Overall · 250 held-out tasks · equal-weight platform macro average
Overall · Estimated API cost per task
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 1m context · 128k max output tokens · estimated mean API cost per task · 90% cache reads assumed
Overall · Programmatic + visual mean
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 1m context · 128k max output tokens · Overall · 250 held-out tasks · equal-weight platform macro average
Overall · Programmatic assertion score
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 1m context · 128k max output tokens · Overall · 250 held-out tasks · equal-weight platform macro average
Overall · Visual assertion score
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 1m context · 128k max output tokens · Overall · 250 held-out tasks · equal-weight platform macro average
Ubuntu · Applications with Prog =100%
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 1m context · 128k max output tokens · Ubuntu · nominal 50-task split · platform-specific score
Ubuntu · Applications with Prog ≥90%
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 1m context · 128k max output tokens · Ubuntu · nominal 50-task split · platform-specific score
Ubuntu · Programmatic assertion score
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 1m context · 128k max output tokens · Ubuntu · nominal 50-task split · platform-specific score
Ubuntu · Visual assertion score
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 1m context · 128k max output tokens · Ubuntu · nominal 50-task split · platform-specific score
Web · Applications with Prog =100%
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 1m context · 128k max output tokens · Web · nominal 50-task split · platform-specific score
Web · Applications with Prog ≥90%
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 1m context · 128k max output tokens · Web · nominal 50-task split · platform-specific score
Web · Programmatic assertion score
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 1m context · 128k max output tokens · Web · nominal 50-task split · platform-specific score
Windows · Applications with Prog =100%
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 1m context · 128k max output tokens · Windows · nominal 50-task split · platform-specific score
Windows · Applications with Prog ≥90%
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 1m context · 128k max output tokens · Windows · nominal 50-task split · platform-specific score
Windows · Programmatic assertion score
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 1m context · 128k max output tokens · Windows · nominal 50-task split · platform-specific score
Windows · Visual assertion score
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 1m context · 128k max output tokens · Windows · nominal 50-task split · platform-specific score
RiemannBenchFable/Mythos 5 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted · Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
SkillsBench8 related benchmarks · 9 results
1.1 · With Skills
Reported by SkillsBench / BenchFlow · via BenchBoard docs
OpenHands · 87 tasks · 3 trials/task · max reasoning effort · with Skills
Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Qwen他测:v1.1 · 87 tasks · 每题3次均值;Opus/Fable=Claude Code,GPT=Codex,Qwen=OpenCode
1.1 · Without Skills
Reported by SkillsBench / BenchFlow · via BenchBoard docs
OpenHands · 87 tasks · 3 trials/task · max reasoning effort · without Skills
Vals · With Skills · Accuracy
Reported by Vals AI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Vals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · With Skills · system=anthropic/claude-opus-4-8 · provider=Anthropic · compute_effort=max · temperature=1 · max_output_tokens=128000
Vals · With Skills · Mean cost / test
Reported by Vals AI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Vals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · With Skills · system=anthropic/claude-opus-4-8 · provider=Anthropic · compute_effort=max · temperature=1 · max_output_tokens=128000
Vals · With Skills · Mean duration / test
Reported by Vals AI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Vals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · With Skills · system=anthropic/claude-opus-4-8 · provider=Anthropic · compute_effort=max · temperature=1 · max_output_tokens=128000
Vals · Without Skills · Accuracy
Reported by Vals AI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Vals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · Without Skills · system=anthropic/claude-opus-4-8 · provider=Anthropic · compute_effort=max · temperature=1 · max_output_tokens=128000
Show 2 more resultsHide 2 results
Vals · Without Skills · Mean cost / test
Reported by Vals AI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Vals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · Without Skills · system=anthropic/claude-opus-4-8 · provider=Anthropic · compute_effort=max · temperature=1 · max_output_tokens=128000
Vals · Without Skills · Mean duration / test
Reported by Vals AI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Vals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · Without Skills · system=anthropic/claude-opus-4-8 · provider=Anthropic · compute_effort=max · temperature=1 · max_output_tokens=128000
Toolathlon VerifiedReported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Vals12 related benchmarks · 14 results
EMB
Source: Vals AI
CorpFin v2
Source: Vals AI
LegalBench
Source: Vals AI
MedCode
Source: Clinical Benchmarks data docs · Sep 2026
Setting not stated
Source: Vals AI
Setting not stated
MedScribe
Source: Clinical Benchmarks data docs · Sep 2026
Setting not stated
Source: Vals AI
Setting not stated
Show 6 more resultsHide 6 results

Data credits: BenchBoard by SYuan03 (github.com/SYuan03/benchboard). MIT covers its code and data organization; no results licence is inferred. Numeric relays retain the original stating-source links. · Epoch AI Benchmarking Hub, CC BY 4.0 (licence) · LMArena, CC BY 4.0 (licence) · Baromètre de l’écriture IA, Johannes Eva; public numeric ranking data, © 2026 Johannes Eva, no results licence stated · Data by LLM Stats · BullshitBench by Peter Gostev, MIT (licence) · ChemBench leaderboard, Jablonka group / lamalab-org. Public verified numeric result summaries; results licence not separately stated. · Clinical Benchmarks (clinicalbenchmarks.ai), CC BY 4.0; fractions converted to percentages; source documents retain their own terms (licence) · EQ-Bench (Samuel J. Paech) — cite paech2023eqbench · Steel.dev Agent Leaderboards, MIT (licence) · Keelemudelite mõõdupuu: Independent LLM leaderboard for Estonian. Eesti Keele Instituut. https://moodupuu.eki.ee, CC BY 4.0. Latest per-benchmark results selected with partial-run flags retained. (licence) · ExploitGym leaderboard (Wang et al., UC Berkeley), arXiv:2605.11086. No licence is stated for the result data; the repository's Apache-2.0 licence covers its code only, and task data keeps its upstream licences. Literal exploit counts; hidden rows omitted. · OpenRouter public benchmark leaderboard and search-lane exports, CC BY 4.0. Fractional source scores converted to percentages; immutable dated snapshots. (licence) · Horangi 4, Weights & Biases Korea; public publisher-selected numeric evaluation results · MCPMark (eval-sys), mcpmark.ai Verified leaderboard. Single-run Pass@1 fractions shown as percentages. · MineBench, MIT (licence) · Nejumi Leaderboard 4, Weights & Biases Japan; public publisher-selected numeric evaluation results · OpenHands Index (All Hands AI), Apache-2.0 (licence) · Hugging Face official benchmark leaderboards; licence varies by benchmark · SnakeBench (Greg Kamradt / ARC Prize Foundation), MIT (licence) · SWE-bench-Live (Microsoft and contributors), swe-bench-live.github.io. Resolve rates computed from the published report counts. · Sierra Research tau2-bench leaderboard submissions (github.com/sierra-research/tau2-bench), MIT (licence)

Also known as: aihubmix/claude-opus-4-8-think, anthropic-claude-opus-4.8, anthropic.claude-opus-4-8, anthropic.claude-opus-4-8-v1, anthropic/claude-opus-4-8, anthropic/claude-opus-4.8, anthropic/claude-opus-4.8-batch, anthropic/claude-opus-4.8-fast, anthropic/claude-opus-4.8:batch, anthropic/claude-opus-4.8:thinking, apac.anthropic.claude-opus-4-8, au.anthropic.claude-opus-4-8, aws-bedrock/claude-opus-4-8, azure_ai/claude-opus-4-8, bedrock/claude-opus-4-8, bedrock/claude-opus-4-8@ap-northeast-1, bedrock/claude-opus-4-8@eu-central-1, bedrock/claude-opus-4-8@eu-north-1, bedrock/claude-opus-4-8@eu-west-1, bedrock/claude-opus-4-8@eu-west-3, bedrock/us-gov-east-1/anthropic.claude-opus-4-8, bedrock/us-gov-west-1/anthropic.claude-opus-4-8, bedrock_mantle/us-gov-west-1/anthropic.claude-opus-4-8, claude-4.8-opus, claude-opus-4-8, claude-opus-4-8-think, claude-opus-4-8-thinking, claude-opus-4-8@default, claude-opus-4-8@eu, claude-opus-4.8-high, claude-opus4-8, cometapi-opus-4-8, databricks/databricks-claude-opus-4-8, deepinfra/anthropic/claude-opus-4-8, eu.anthropic.claude-opus-4-8, eu.anthropic.claude-opus-4-8-v1, github_copilot/claude-opus-4.8, github_copilot/claude-opus-4.8-fast, global.anthropic.claude-opus-4-8, global.anthropic.claude-opus-4-8-v1, global.anthropic.claude-opus-4-8-v1:0, jp.anthropic.claude-opus-4-8, perplexity/anthropic/claude-opus-4-8, regional.anthropic.claude-opus-4-8-v1:0, snowflake/claude-opus-4-8, us-gov.anthropic.claude-opus-4-8, us.anthropic.claude-opus-4-8, us.anthropic.claude-opus-4-8-v1, vertex/claude-opus-4-8, vertex/claude-opus-4-8@eu, vertex/claude-opus-4-8@us, vertex_ai/claude-opus-4-8, vertex_ai/claude-opus-4-8@default

Like this one. Tap or hover any dotted value, label or icon to see where it comes from and how it is measured. open details when tapped or hovered. How this data works