Skip to content

OpenAI · Chat

GPT-5.6 Sol

OpenAI 🇺🇸ChatGPT familyOne of its sources says the weights aren't publicly available.The maker currently lists 11 Jan 2028 as the deprecation date. Plans can change.

Newer version: GPT-6.2 Sol. We compared release dates and version names in our catalogue.

From Venice's listing:
GPT-5.6 Sol is the flagship model in OpenAI's GPT-5.6 series. It is suited for complex reasoning, coding, and agentic workflows, and is particularly strong at command-line and multi-step coding tasks and long-horizon problem solving.
llminfo IQ
141.6
Maker's price
$8.00 blended / 1M
Max context
1.1M
Listings
124

Evidence and sources

llminfo IQ 141.6, ranked among chat models. From 504 benchmarks. Plenty of data behind it.How llminfo IQ is computedCoding IQ 125.1, 95% interval 120–130. From 38 benchmark families across 16 sources. Plenty of data behind it. Uses the model's best published results. It has its own scale, separate from llminfo IQ.How Coding IQ is computedGood value (59.9). Cheaper per task than its benchmark results predict. From 69 benchmarks with measured task costs. Plenty of data behind it.How Value for money worksYap score: tokens used per coding task vs. models with the same resultsTerse (41 ±8). Generates fewer tokens than models with similar coding results. Measured at max effort. A fair amount of data behind it. By effort: default 39 · low 34 · medium 38 · high 41 · max 41.Median 40,745 output tokens per task at max effort.How the Yap score worksReleased 26 Jun 2026 Training-data cutoff, as its sources state it Maker HQ: US (according to OpenAI's own website)
All rankings: #15 in Google Trends interest this week #67 on OpenRouter this month #88 on OpenRouter this week #95 on OpenRouter today #285 on NanoGPT recentlyAll rankings: #15 in Google Trends interest this week #67 on OpenRouter this month #88 on OpenRouter this week #95 on OpenRouter today #285 on NanoGPT recentlySources: OpenAI API data, Writer Pricing, LiteLLM data, Models.dev data, AI/ML API data, OpenRouter data docs, Venice.ai data, llminfo researchPrimary data from OpenAI API from Writer Pricing: name, variant from LiteLLM: deprecation date from Models.dev: weights availability from AI/ML API: primary mode from OpenRouter: knowledge cutoff from Venice.ai: description from llminfo research: release date
Tags from its sources:instructreasoningtoolsvisiondocumentlong-contextfrontieragenticcoding

Advantages and trade-offs

Compared with 10 chat models, picked mainly by popularity and IQ

Advantages

Trade-offs

No open weights

The weights aren't public, so you can't run it yourself.

We only make a claim when there's enough data on models of the same type. Updated 22 Nov, 04:34 UTC.Prices come from the model's maker where they sell it, otherwise their official partner, otherwise the market median. This comparison uses 10 maker's prices.

Input & output

as stated by the model's providers
Text
Images
Files
Text

Capabilities

Supported operations and provider-stated features

VisionReasoningFunction callingStructured outputPrompt cachingPDF inputComputer useWeb searchBatch API

Listings

 OpenAIgpt-5.6-solSources: GenAI Prices data, LiteLLM data, Models.dev data, OpenAI API data922K$5.00cache read $0.50batch $2.00$30.00cache write $6.25batch $10.00———
 Xxapigpt-5.6-solSource: Models.dev data1.1M$0.2462cache read $0.0246$1.231cache write $0.3077———
 Jzstokengpt-5.6-solSource: Models.dev data1.1M$0.6228cache read $0.0623$2.80cache write $0.7784———
 Koroutergpt-5.6-solSource: Models.dev data1.1M$0.80cache read $0.08$4.00cache write $1.00———
 Xpersonagpt-5.6-solSources: Models.dev data, Xpersona data372K$1.50cache read $0.15$12.00$0.001——
 Nous Portalopenai/gpt-5.6-sol:batch · batchSource: Nous Portal Inference Gateway data1.1M$2.00cache read $0.20$10.00cache write $2.50———
 OpenRouterInference by OpenAIopenai/gpt-5.6-sol · flexSource: OpenRouter data docs922K$2.00cache read $0.20$10.00cache write $2.50—145.53,833.5ms
 Replicateopenai/gpt-5.6-solSources: LiteLLM data, Replicate data docsNot published by the provider$2.00$10.00———
 Vercelopenai/gpt-5.6-sol-batch · batchSource: Vercel AI Gateway data1.1M$2.00cache read $0.20$10.00———
 Boundlessgpt-5.6-solSource: Models.dev data262K$2.00cache read $0.20$10.00———

17 listings without a price are hidden.

Included in plans

Copilot EnterprisegithubSubscription$39/moextra use billed past the allowancePlan page
Copilot Pro+githubSubscription$39/moextra use billed past the allowancePlan page
Copilot BusinessgithubSubscription$19/moextra use billed past the allowancePlan page
Copilot MaxgithubSubscription$100/moextra use billed past the allowancePlan page
ChatGPT PlusOpenAISubscription$20/moPlan page
ChatGPT ProOpenAISubscription$100/moPlan page

Benchmarks

Named effort levels run from lowest to highest. Select a result to see its source details.

Agentic

Agentic IF Indexmax · Meta internal composite · agentic instruction following · Reported by Meta · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Agents' Last Exam3 related benchmarks · 11 results
Agents' Last Exam
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Overall Score
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Reasoning effort
OpenAI GPT-5.6 launch table
OpenAI launch table · highest published effort
Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Qwen表:Overall Score
Pass Rate
Reported by DeepSeek · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
DeepSeek V4.1 launch chart · settings unspecified
Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Arena AgentSource: LMArena data
BrowseComp18 results
Reported by Shanghai AI Laboratory · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Atria Dawn Preview official card / report Table 1 · published comparison run · model snapshots and reasoning tiers not specified unless stated · in-house search/visit/python harness · 500 steps · discard-all context strategy
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI GPT-5.6 launch table
OpenAI launch table
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Opus 5 System Card capability summary · adaptive thinking · max effort unless noted
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
Show 12 more resultsHide 12 results
Source: OpenRouter benchmark exports · Aug 2026
exa search · OpenRouter's search tool · up to 1 turn · high effort
exa search · OpenRouter's search tool · up to 25 turns · high effort
exa search · OpenRouter's search tool · up to 5 turns · high effort
the model's own search · OpenRouter's search tool · up to 1 turn · high effort
the model's own search · OpenRouter's search tool · up to 25 turns · high effort
the model's own search · OpenRouter's search tool · up to 5 turns · high effort
parallel search · OpenRouter's search tool · up to 1 turn · high effort
parallel search · OpenRouter's search tool · up to 25 turns · high effort
parallel search · OpenRouter's search tool · up to 5 turns · high effort
perplexity search · OpenRouter's search tool · up to 1 turn · high effort
perplexity search · OpenRouter's search tool · up to 25 turns · high effort
perplexity search · OpenRouter's search tool · up to 5 turns · high effort
DeepResearch Bench IIAtria Dawn Preview official card / report Table 1 · published comparison run · model snapshots and reasoning tiers not specified unless stated · Claude Code · identical search tools · four-hour task limit · GPT-5.5 judge · three independent scoring passes · Reported by Shanghai AI Laboratory · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
GDPval · Atria paper EloAtria Dawn Preview official card / report Table 1 · published comparison run · model snapshots and reasoning tiers not specified unless stated · 220 tasks · AgentCompass / OpenHands · four-hour limit · 250 steps · GLM-5.3 rubric judge · pairwise Elo · Reported by Shanghai AI Laboratory · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
GDPval-AA v27 results
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Anthropic Fable/Mythos 5.1 launch table
Opus 5 System Card capability summary · adaptive thinking · max effort unless noted
System Card capability summary · adaptive thinking · max effort · default sampling · 5 trials · context ≤1M unless noted
Reported by Z.ai · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Artificial Analysis / Z.ai转载
Reported by Google DeepMind · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Reported by Meta · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
max · Artificial Analysis Stirrup · 220 tasks · shell + web browsing · human baseline 1000
Show 1 more resultHide 1 result
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI GPT-5.6 launch table
JobBench3 results
Reported by Shanghai AI Laboratory · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Atria Dawn Preview official card / report Table 1 · published comparison run · model snapshots and reasoning tiers not specified unless stated · official OpenCode harness · main split / 65 tasks · 7,200-second limit · two attempts · Grok-4.3 judge · mean task score
Reported by Meta · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
max · 65 tasks · official evaluation code · OpenCode · file-aware rubric grader · mean rubric score
Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
MCPMark VerifiedSource: MCPMark Verified leaderboard docs
OSWorld4 related benchmarks · 13 results
OSWorld 2
Source: Epoch AI Benchmarking Hub data
Source: Steel.dev Agent Leaderboards data · Jul 2026
Setting not stated
2.0 · Binary
Reported by Meta · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
max · OSWorld 2.0 v08.08 · 108 tasks · common internal framework · binary completion
2.0 · Partial
Reported by Meta · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
max · OSWorld 2.0 v08.08 · 108 tasks · common internal framework · partial reward
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Reasoning effort · OSWorld 2.0 offline v2026.08.08
v2026.08.08 · offline set · partial score
OSWorld 2.0
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Opus 5 System Card setting
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OSWorld 2.0 · OpenAI GPT-5.6 launch table
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI GPT-5.6 launch table
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
SWE-Atlas · Codebase Q&Amax · provider model card · public QnA split · mean pass@1 · Reported by Meta · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
SWE-Bench Pro6 results
Reported by Shanghai AI Laboratory · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Atria Dawn Preview official card / report Table 1 · published comparison run · model snapshots and reasoning tiers not specified unless stated · OpenHands v1.23.0 · 4.5-hour limit · 250 interaction steps · network disabled and Git history removed
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI GPT-5.6 launch table
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Opus 5 System Card capability summary · adaptive thinking · max effort unless noted
System Card capability summary · adaptive thinking · max effort · default sampling · 5 trials · context ≤1M unless noted
Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Qwen表:Claude Code · temp 1.0 · top_p 0.95 · 256K · refined tasks
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
Source: OpenRouter benchmark exports · Oct 2026
9 Jul 2026 version (gpt-5.6-sol) · 90-day results
Across 3 providers lowest 72.5% median 73.1% highest 74.2%OpenRouter's automatic routing 73.3%
Show results from 3 providers and OpenRouter's automatic routingHide results from 3 providers and OpenRouter's automatic routing
  • Amazon Bedrock
  • OpenRouter's automatic routing
  • OpenAI
  • Azure
9 Jul 2026 version (gpt-5.6-sol-pro) · 90-day results · pro
Across 2 providers lowest 67.3% median 69.5% highest 71.7%OpenRouter's automatic routing 70.7%
Show results from 2 providers and OpenRouter's automatic routingHide results from 2 providers and OpenRouter's automatic routing
  • OpenAI
  • OpenRouter's automatic routing
  • Azure
Terminal-Bench 2.15 related benchmarks · 30 results
Terminal-Bench 2.1
Reported by Shanghai AI Laboratory · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Atria Dawn Preview official card / report Table 1 · published comparison run · model snapshots and reasoning tiers not specified unless stated · four-run mean · 12× timeout multiplier capped at 12 hours · 500 turns · Claude Code v2.1.224 except GPT models use Terminus 2
Reported by DeepSeek · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
DeepSeek V4.1 launch chart · settings unspecified
Reported by Google DeepMind · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Reported by Meta · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
max · 89 tasks · native coding harness · isolated cloud sandbox · pass@1
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI GPT-5.6 launch table
Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Qwen表:Qwen 采用 Claude Code avg@10、5h、max_tokens 131072;友商取公开最高分
Reported by Z.ai, DeepSeek · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
Terminal-Bench 3.0
Reported by Z.ai · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Setting not stated
Reported by DeepSeek · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
DeepSeek Harness / max effort
DeepSeek V4.1 launch chart · settings unspecified
Terminal-Bench 4.0
Reported by Terminal-Bench / Stanford / Harbor / Laude Institute · via BenchBoard docs
Terminal-Bench 4.0 official leaderboard · Codex · max effort
Source: LLM Stats data
Setting not stated
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Anthropic Fable/Mythos 5.1 launch table
System Card capability summary · adaptive thinking · max effort · default sampling · 5 trials · context ≤1M unless noted
Reported by DeepSeek · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
DeepSeek Harness / max effort
DeepSeek V4.1 launch chart · settings unspecified
Reported by Google DeepMind · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Reported by SpaceXAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Harbor · provider harness · task success rate · max effort
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Reasoning effort · Non-code
Non-code · max effort · chart displays 37.3% (raw 37.2727%)
OpenAI launch table
Terminal-Bench Science 0.1
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Claude Code harness
System Card capability summary · adaptive thinking · max effort · default sampling · 5 trials · context ≤1M unless noted
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI launch table

Reasoning

ARC-AGI4 related benchmarks · 19 results
ARC-AGI
Source: Epoch AI Benchmarking Hub data
Reasoning effort
ARC-AGI 2
Source: Epoch AI Benchmarking Hub data
Reasoning effort
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI launch table
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Opus 5 System Card capability summary · adaptive thinking · max effort unless noted
System Card capability summary · adaptive thinking · max effort · default sampling · 5 trials · context ≤1M unless noted
ARC-AGI-1
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI launch table
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
System Card capability summary · adaptive thinking · max effort · default sampling · 5 trials · context ≤1M unless noted
ARC-AGI-3
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI GPT-5.6 launch table
OpenAI launch table
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
AutomationBench15 results
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Anthropic Fable/Mythos 5.1 launch table
Opus 5 System Card capability summary · adaptive thinking · max effort unless noted
System Card capability summary · adaptive thinking · max effort · default sampling · 5 trials · context ≤1M unless noted
Reported by DeepSeek · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
DeepSeek V4.1 launch chart · settings unspecified
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Reasoning effort
OpenAI GPT-5.6 launch table
Show 5 more resultsHide 5 results
OpenAI launch table
Reported by Meta · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
max · public v3 · 600 tasks · benchmark automation tools · pass@1
Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Qwen表:600题公开子集 · Pass@1
Reported by Z.ai · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
AutomationBench v1.0.6Atria Dawn Preview official card / report Table 1 · published comparison run · model snapshots and reasoning tiers not specified unless stated · v1.0.6 with null-type fix from PR #13 · candidate harness and task count unstated · Reported by Shanghai AI Laboratory · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
BabyVisionDeepSeek V4.1 launch chart · settings unspecified · with tools · Reported by DeepSeek · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI GPT-5.6 launch table
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
ChartographyDeepSeek V4.1 launch chart · settings unspecified · with tools · Reported by DeepSeek · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
CoWorkBenchReported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
DeepSearchQA13 results
Reported by Meta · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
max · 900 questions · common browser search/open/find harness · semantic answer matching
Show 12 more resultsHide 12 results
Source: OpenRouter benchmark exports · Aug 2026
exa search · OpenRouter's search tool · up to 1 turn · high effort
exa search · OpenRouter's search tool · up to 25 turns · high effort
exa search · OpenRouter's search tool · up to 5 turns · high effort
the model's own search · OpenRouter's search tool · up to 1 turn · high effort
the model's own search · OpenRouter's search tool · up to 25 turns · high effort
the model's own search · OpenRouter's search tool · up to 5 turns · high effort
parallel search · OpenRouter's search tool · up to 1 turn · high effort
parallel search · OpenRouter's search tool · up to 25 turns · high effort
parallel search · OpenRouter's search tool · up to 5 turns · high effort
perplexity search · OpenRouter's search tool · up to 1 turn · high effort
perplexity search · OpenRouter's search tool · up to 25 turns · high effort
perplexity search · OpenRouter's search tool · up to 5 turns · high effort
GDP.pdf2 results
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI GPT-5.6 launch table
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
GeneBench Pro3 results
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI GPT-5.6 launch table
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
HLE12 results
Show 12 more resultsHide 12 results
Source: OpenRouter benchmark exports · Aug 2026
exa search · OpenRouter's search tool · up to 1 turn · high effort
exa search · OpenRouter's search tool · up to 25 turns · high effort
exa search · OpenRouter's search tool · up to 5 turns · high effort
the model's own search · OpenRouter's search tool · up to 1 turn · high effort
the model's own search · OpenRouter's search tool · up to 25 turns · high effort
the model's own search · OpenRouter's search tool · up to 5 turns · high effort
parallel search · OpenRouter's search tool · up to 1 turn · high effort
parallel search · OpenRouter's search tool · up to 25 turns · high effort
parallel search · OpenRouter's search tool · up to 5 turns · high effort
perplexity search · OpenRouter's search tool · up to 1 turn · high effort
perplexity search · OpenRouter's search tool · up to 25 turns · high effort
perplexity search · OpenRouter's search tool · up to 5 turns · high effort
Reported by DeepSeek · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
DeepSeek V4.1 launch chart · settings unspecified · full set
Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
LifeSciBench3 results
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI GPT-5.6 launch table
OpenAI launch table
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI GPT-5.6 launch table
Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Qwen表:Verified · Pass@1
Reported by Z.ai · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
WideSearch12 results
Show 12 more resultsHide 12 results
Source: OpenRouter benchmark exports · Aug 2026
exa search · OpenRouter's search tool · up to 1 turn · high effort
exa search · OpenRouter's search tool · up to 25 turns · high effort
exa search · OpenRouter's search tool · up to 5 turns · high effort
the model's own search · OpenRouter's search tool · up to 1 turn · high effort
the model's own search · OpenRouter's search tool · up to 25 turns · high effort
the model's own search · OpenRouter's search tool · up to 5 turns · high effort
parallel search · OpenRouter's search tool · up to 1 turn · high effort
parallel search · OpenRouter's search tool · up to 25 turns · high effort
parallel search · OpenRouter's search tool · up to 5 turns · high effort
perplexity search · OpenRouter's search tool · up to 1 turn · high effort
perplexity search · OpenRouter's search tool · up to 25 turns · high effort
perplexity search · OpenRouter's search tool · up to 5 turns · high effort

Coding

Arena ELOSource: LMArena data · Nov 2026
BenchCAD4 results
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI launch table
Python tool enabled
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
Reported by Cursor · via BenchBoard docs
Reasoning effort · Cursor production agent
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Cursor production agent · max effort
CyberGym3 results
Reported by Shanghai AI Laboratory · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Atria Dawn Preview official card / report Table 1 · published comparison run · model snapshots and reasoning tiers not specified unless stated · Claude Code v2.1.119 · high thinking effort · four-hour-ten-minute limit · 2 CPUs / 4 GB
Reported by Z.ai · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Claude Code / 无Web工具
Reported by DeepSeek · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
DeepSeek V4.1 launch chart · settings unspecified
ExploitBench4 results
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI GPT-5.6 launch table
OpenAI launch table
Reported by Z.ai · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
FrontierCode 1.13 related benchmarks · 11 results
FrontierCode 1.1
Source: FrontierCode data
Source: LLM Stats data
Setting not stated
Extended
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI launch table
Main
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Reasoning effort
OpenAI launch table
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Opus 5 System Card capability summary · adaptive thinking · max effort unless noted
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
adaptive thinking · max effort · default sampling · 5-trial mean · context ≤1M unless noted · Proximal public result · max · mean@5
Opus 5.5 System Card capability section
Reported by SpaceXAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Proximal Proximus harness · mean@5 · max effort
KernelGen 1P2 results
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI GPT-5.6 launch table
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
MLS-Bench-LiteQwen表:Claude Code · 5h · max_tokens 131072;对照来自官方榜 · Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
NanoGPT2 results
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI GPT-5.6 launch table
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
PaperBenchQwen表:BasicAgent · Code-Dev · Claude Opus 4.6 judge · 3次均值 · 最长12h · Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
QwenSVGBenchQwen内部双语 SVG 基准 · 自动渲染 + 多模态评判 · BT/Elo · Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
QwenSWEBenchQwen内部基准:Claude Code · avg@3 · 8h · 256K · Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
RSI Index2 results
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI GPT-5.6 launch table
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
SEC-Bench Pro5 results
Reported by DeepSeek · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
DeepSeek V4.1 launch chart · settings unspecified
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI GPT-5.6 launch table
OpenAI launch table
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
SkillsBenchLing-3.1-flash official chart · SkillsBench · version, task count, harness, reasoning setting and run count not stated · Reported by Ant inclusionAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
SWE-bench Verified4 related benchmarks · 6 results
SWE-bench-Live Lite
Source: SWE-bench-Live leaderboard data
By scaffold · set: lite · instances: 300
SWE-bench-Live MultiLang Java
Source: SWE-bench-Live leaderboard data
By scaffold · set: java · instances: 160
SWE-bench-Live MultiLang TypeScript/JavaScript
Source: SWE-bench-Live leaderboard data · Sep 2026
scaffold: slingshot-v3.4.0, set: tsjs, instances: 219
ValsSource: Vals AI

Human preference

Arena ELO64 related benchmarks · 64 results
Source: LMArena data · Nov 2026
BenchmarkxHigh
Arena ELO
Chinese
Creative Writing
Document
Show 60 more benchmarksHide 60 benchmarks
BenchmarkxHigh
Document Style-Controlled
English
Exclude Ties
Expert
French
German
Hard Prompts
Hard Prompts English
Industry Business and Management and Financial Operations
Industry Entertainment and Sports and Media
Industry Life and Physical and Social Science
Industry Mathematical
Industry Medicine and Healthcare
Industry Software and It Services
Industry Writing and Literature and Language
Japanese
Korean
Longer Query
Multi Turn
Non English
Polish
Russian
Search Factuality
Search Style-Controlled
Spanish
Style-Controlled
Text Factuality
Text Factuality - Chinese
Text Factuality - Creative Writing
Text Factuality - English
Text Factuality - Exclude Ties
Text Factuality - Expert
Text Factuality - French
Text Factuality - Hard Prompts
Text Factuality - Hard Prompts English
Text Factuality - Industry Business and Management and Financial Operations
Text Factuality - Industry Entertainment and Sports and Media
Text Factuality - Industry Life and Physical and Social Science
Text Factuality - Industry Mathematical
Text Factuality - Industry Medicine and Healthcare
Text Factuality - Industry Software and It Services
Text Factuality - Industry Writing and Literature and Language
Text Factuality - Korean
Text Factuality - Longer Query
Text Factuality - Multi Turn
Text Factuality - Non English
Text Factuality - Polish
Text Factuality - Russian
Text Factuality - Spanish
Vision
Vision - Chinese
Vision - Creative Writing Vision
Vision - Diagram
Vision - English
Vision - Homework
Vision - Humor
Vision Style-Controlled

General

Multilingual

EKI EstonianSource: Keelemudelite mõõdupuu data docs
NepNewsBench v2 draft40 related benchmarks · 40 results
entity F1 (all questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
OpenAI via openrouter, reasoning off
core fact recall (all questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
OpenAI via openrouter, reasoning off
core fact recall (English-source questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
OpenAI via openrouter, reasoning off
core fact recall (Nepali-source questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
OpenAI via openrouter, reasoning off
coverage precision proxy (all questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
OpenAI via openrouter, reasoning off
Show 34 more resultsHide 34 results
effective score (all questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
OpenAI via openrouter, reasoning off
effective score (English-source questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
OpenAI via openrouter, reasoning off
effective score (Nepali-source questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
OpenAI via openrouter, reasoning off
English prose quality (all questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
OpenAI via openrouter, reasoning off
entity F1 (English-source questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
OpenAI via openrouter, reasoning off
entity F1 (Nepali-source questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
OpenAI via openrouter, reasoning off
entity precision (all questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
OpenAI via openrouter, reasoning off
entity precision (English-source questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
OpenAI via openrouter, reasoning off
entity precision (Nepali-source questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
OpenAI via openrouter, reasoning off
entity recall (all questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
OpenAI via openrouter, reasoning off
entity recall (English-source questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
OpenAI via openrouter, reasoning off
entity recall (Nepali-source questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
OpenAI via openrouter, reasoning off
Nepali prose quality (all questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
OpenAI via openrouter, reasoning off
supporting fact recall (all questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
OpenAI via openrouter, reasoning off
supporting fact recall (Nepali-source questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
OpenAI via openrouter, reasoning off
valid-answer quality (all questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
OpenAI via openrouter, reasoning off
valid-answer quality (Nepali-source questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
OpenAI via openrouter, reasoning off

Science

GPQASource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
GPQA Diamond15 results
Source: Epoch AI Benchmarking Hub data
Reasoning effort
Reported by DeepSeek · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
DeepSeek V4.1 launch chart · settings unspecified
Pass@1 / max effort
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI GPT-5.6 launch table
OpenAI launch table
Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Source: OpenRouter benchmark exports · Oct 2026
9 Jul 2026 version (gpt-5.6-sol) · 90-day results
Across 3 providers lowest 92.2% median 92.6% highest 92.9%OpenRouter's automatic routing 92.4%
Show results from 3 providers and OpenRouter's automatic routingHide results from 3 providers and OpenRouter's automatic routing
  • Azure
  • OpenAI
  • OpenRouter's automatic routing
  • Amazon Bedrock
9 Jul 2026 version (gpt-5.6-sol-pro) · 90-day results · pro
Across 2 providers lowest 93.9% median 93.9% highest 93.9%OpenRouter's automatic routing 94.1%
Show results from 2 providers and OpenRouter's automatic routingHide results from 2 providers and OpenRouter's automatic routing
  • OpenRouter's automatic routing
  • Azure
  • OpenAI
MedChemBench2 results
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
internal · weighted score
ValsSource: Vals AI

Math

FrontierMath5 related benchmarks · 9 results
FrontierMath
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Tier 1-3 (v2)
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Tier 4 (v2)
Source: Epoch AI Benchmarking Hub data
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
v2 · GPT-5.6 launch snapshot · 2026-07-09
v2 · GPT-6 Astra launch snapshot · 2026-09-01
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated

Long context

LongBench v2Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Reported by Meta · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
max · 100 examples · 8-needle · no tools · mean sequence-matcher ratio

Vision

LVBenchGemini为agentic;跨模型设置见原卡 · Reported by Google DeepMind · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
MMMU-Pro3 results
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated

Factuality

Instruction following

IFBenchReported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.

Knowledge

OCR

Safety

Speech

Other

AndroidBenchQwen表:95题公开子集 · avg@3 · Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Opus 5 System Card capability summary · adaptive thinking · max effort unless noted
System Card capability summary · adaptive thinking · max effort · default sampling · 5 trials · context ≤1M unless noted
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Version · Artificial Analysis Intelligence Index
BlenderWorld15 related benchmarks · 15 results
Art & Animation
Reported by BlenderWorld authors / Alibaba Token Hub · via BenchBoard docs
benchmark-owner paper run · 30 category tasks · Blender 4.2.10 / Blender MCP paper setup · GPT-5.6 Terra main judge · generation-agent harness and reasoning effort not stated on public leaderboard
Architecture & Interiors
Reported by BlenderWorld authors / Alibaba Token Hub · via BenchBoard docs
benchmark-owner paper run · 72 category tasks · Blender 4.2.10 / Blender MCP paper setup · GPT-5.6 Terra main judge · generation-agent harness and reasoning effort not stated on public leaderboard
Characters & Animals
Reported by BlenderWorld authors / Alibaba Token Hub · via BenchBoard docs
benchmark-owner paper run · 25 category tasks · Blender 4.2.10 / Blender MCP paper setup · GPT-5.6 Terra main judge · generation-agent harness and reasoning effort not stated on public leaderboard
Fully satisfied rubric
Reported by BlenderWorld authors / Alibaba Token Hub · via BenchBoard docs
benchmark-owner paper run · 223 tasks · Blender 4.2.10 / Blender MCP paper setup · GPT-5.6 Terra main judge · generation-agent harness and reasoning effort not stated on public leaderboard
Image/video-input overall score
Reported by BlenderWorld authors / Alibaba Token Hub · via BenchBoard docs
benchmark-owner paper run · 145 image or video reference tasks · Blender 4.2.10 / Blender MCP paper setup · GPT-5.6 Terra main judge · generation-agent harness and reasoning effort not stated on public leaderboard
Nature & Landscapes
Reported by BlenderWorld authors / Alibaba Token Hub · via BenchBoard docs
benchmark-owner paper run · 21 category tasks · Blender 4.2.10 / Blender MCP paper setup · GPT-5.6 Terra main judge · generation-agent harness and reasoning effort not stated on public leaderboard
Show 9 more resultsHide 9 results
Overall score
Reported by BlenderWorld authors / Alibaba Token Hub · via BenchBoard docs
benchmark-owner paper run · 223 tasks · Blender 4.2.10 / Blender MCP paper setup · GPT-5.6 Terra main judge · generation-agent harness and reasoning effort not stated on public leaderboard
Procedural & Game Assets
Reported by BlenderWorld authors / Alibaba Token Hub · via BenchBoard docs
benchmark-owner paper run · 10 category tasks · Blender 4.2.10 / Blender MCP paper setup · GPT-5.6 Terra main judge · generation-agent harness and reasoning effort not stated on public leaderboard
Products & Manufacturing
Reported by BlenderWorld authors / Alibaba Token Hub · via BenchBoard docs
benchmark-owner paper run · 28 category tasks · Blender 4.2.10 / Blender MCP paper setup · GPT-5.6 Terra main judge · generation-agent harness and reasoning effort not stated on public leaderboard
Rendering & Simulation
Reported by BlenderWorld authors / Alibaba Token Hub · via BenchBoard docs
benchmark-owner paper run · 17 category tasks · Blender 4.2.10 / Blender MCP paper setup · GPT-5.6 Terra main judge · generation-agent harness and reasoning effort not stated on public leaderboard
Science & Visualization
Reported by BlenderWorld authors / Alibaba Token Hub · via BenchBoard docs
benchmark-owner paper run · 9 category tasks · Blender 4.2.10 / Blender MCP paper setup · GPT-5.6 Terra main judge · generation-agent harness and reasoning effort not stated on public leaderboard
Text-only overall score
Reported by BlenderWorld authors / Alibaba Token Hub · via BenchBoard docs
benchmark-owner paper run · 78 text-only tasks · Blender 4.2.10 / Blender MCP paper setup · GPT-5.6 Terra main judge · generation-agent harness and reasoning effort not stated on public leaderboard
Vehicles & Machinery
Reported by BlenderWorld authors / Alibaba Token Hub · via BenchBoard docs
benchmark-owner paper run · 11 category tasks · Blender 4.2.10 / Blender MCP paper setup · GPT-5.6 Terra main judge · generation-agent harness and reasoning effort not stated on public leaderboard
Visual-quality dimension score
Reported by BlenderWorld authors / Alibaba Token Hub · via BenchBoard docs
benchmark-owner paper run · 223 tasks · Blender 4.2.10 / Blender MCP paper setup · GPT-5.6 Terra main judge · generation-agent harness and reasoning effort not stated on public leaderboard
Weighted rubric score
Reported by BlenderWorld authors / Alibaba Token Hub · via BenchBoard docs
benchmark-owner paper run · 223 tasks · Blender 4.2.10 / Blender MCP paper setup · GPT-5.6 Terra main judge · generation-agent harness and reasoning effort not stated on public leaderboard
CADGenBench · GenerationMecado · generation split · provider harnesses · reward · xhigh effort · Reported by SpaceXAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
CursorBench 3.2Anthropic Fable/Mythos 5.1 launch table · Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
DeepSearchQA · Atria paper scoreAtria Dawn Preview official card / report Table 1 · published comparison run · model snapshots and reasoning tiers not specified unless stated · in-house search/visit/python harness · 500 steps · 256K context / 64K max output · score subtype unstated · Reported by Shanghai AI Laboratory · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
DeepSWE v1.115 results
Reported by SpaceXAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Datacurve mini-SWE-agent · Pass@1 · max effort
Reported by DeepSeek · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
DeepSeek V4.1 launch chart · settings unspecified
mini-SWE / DeepSeek设置
Reported by Google DeepMind · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Reasoning effort
OpenAI GPT-5.6 launch table
Show 5 more resultsHide 5 results
OpenAI launch table
Reported by Meta · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
max · official Datacurve leaderboard · task pass rate
Reported by Z.ai · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
mini-swe-agent / Z.ai设置
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Opus 5 System Card capability summary · adaptive thinking · max effort unless noted
Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Qwen表:Claude Code / mini-SWE-agent 取较高值 · 256K
E-Commerce Bench7 related benchmarks · 7 results
Mean final assets
Reported by E-Commerce Bench authors / Alibaba Qwen · via BenchBoard docs
benchmark-owner 365-day merchant simulation · five independent episodes · up to four stores · initial balance ¥100,000 · max reasoning · exact API snapshot not stated for unversioned labels
AnchorRatio · repeat-order pricing
Reported by E-Commerce Bench authors / Alibaba Qwen · via BenchBoard docs
benchmark-owner 365-day merchant simulation · five independent episodes · up to four stores · initial balance ¥100,000 · max reasoning · exact API snapshot not stated for unversioned labels
BadSpend · fraudulent-supplier spend
Reported by E-Commerce Bench authors / Alibaba Qwen · via BenchBoard docs
benchmark-owner 365-day merchant simulation · five independent episodes · up to four stores · initial balance ¥100,000 · max reasoning · exact API snapshot not stated for unversioned labels
Controllable return
Reported by E-Commerce Bench authors / Alibaba Qwen · via BenchBoard docs
benchmark-owner 365-day merchant simulation · five independent episodes · up to four stores · initial balance ¥100,000 · max reasoning · exact API snapshot not stated for unversioned labels
CSE+ · honest-deal negotiation
Reported by E-Commerce Bench authors / Alibaba Qwen · via BenchBoard docs
benchmark-owner 365-day merchant simulation · five independent episodes · up to four stores · initial balance ¥100,000 · max reasoning · exact API snapshot not stated for unversioned labels
Drawdown / peak assets
Reported by E-Commerce Bench authors / Alibaba Qwen · via BenchBoard docs
benchmark-owner 365-day merchant simulation · five independent episodes · up to four stores · initial balance ¥100,000 · max reasoning · exact API snapshot not stated for unversioned labels
Show 1 more resultHide 1 result
Yuan per tool call
Reported by E-Commerce Bench authors / Alibaba Qwen · via BenchBoard docs
benchmark-owner 365-day merchant simulation · five independent episodes · up to four stores · initial balance ¥100,000 · max reasoning · exact API snapshot not stated for unversioned labels
EEBenchAtopile · provider harnesses · reward · max effort · Reported by SpaceXAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
ExploitGym9 related benchmarks · 19 results
Solved Tasks
Reported by Z.ai · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
2-hour normalized budget
6-hour normalized budget
on-target exploits
Source: ExploitGym leaderboard data docs · Jul 2026
By eval · agent: Codex CLI · model note: reasoning max · version: v1 · focus: model
on-target exploits (kernel)
Source: ExploitGym leaderboard data docs · Jul 2026
By eval · agent: Codex CLI · model note: reasoning max · version: v1 · focus: model
on-target exploits (userspace)
Source: ExploitGym leaderboard data docs · Jul 2026
By eval · agent: Codex CLI · model note: reasoning max · version: v1 · focus: model
on-target exploits (V8)
Source: ExploitGym leaderboard data docs · Jul 2026
By eval · agent: Codex CLI · model note: reasoning max · version: v1 · focus: model
on-target exploits with mitigations (kernel)
Source: ExploitGym leaderboard data docs · Jul 2026
By eval · agent: Codex CLI · model note: reasoning max · version: v1 · focus: model
Show 7 more resultsHide 7 results
on-target exploits with mitigations (userspace)
Source: ExploitGym leaderboard data docs · Jul 2026
By eval · agent: Codex CLI · model note: reasoning max · version: v1 · focus: model
on-target exploits with mitigations (V8)
Source: ExploitGym leaderboard data docs · Jul 2026
By eval · agent: Codex CLI · model note: reasoning max · version: v1 · focus: model
Success Rate
Reported by DeepSeek · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
DeepSeek V4.1 launch chart · settings unspecified
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI GPT-5.6 launch table
OpenAI launch table
ExplorationBench22 related benchmarks · 22 results
AlienCode · M4 · fixed-probe control
Reported by Fudan University / Tencent Hy / Tsinghua University · via BenchBoard docs
official 2026-10-13 leaderboard snapshot · 70 held-out tasks in this world · 4 exploration rounds · 3 answers/task · max reasoning · one matched control trajectory
AlienCode · A4+O · complete rules after exploration
Reported by Fudan University / Tencent Hy / Tsinghua University · via BenchBoard docs
official 2026-10-13 leaderboard snapshot · 70 held-out tasks in this world · 4 exploration rounds · 3 answers/task · max reasoning · complete-rule reference or auxiliary diagnostic
AlienCode · M0 · baseline of Best@3 trajectory
Reported by Fudan University / Tencent Hy / Tsinghua University, Tencent Hy / ExplorationBench authors · via BenchBoard docs
official 2026-10-13 leaderboard snapshot · 70 held-out tasks in this world · 4 exploration rounds · 3 answers/task · max reasoning · 3 independent autonomous trajectories
AlienCode · M4 · Best@3 held-out accuracy
Reported by Fudan University / Tencent Hy / Tsinghua University, Tencent Hy / ExplorationBench authors · via BenchBoard docs
official 2026-10-13 leaderboard snapshot · 70 held-out tasks in this world · 4 exploration rounds · 3 answers/task · max reasoning · 3 independent autonomous trajectories
AlienCode · M4 · direct answering control
Reported by Fudan University / Tencent Hy / Tsinghua University · via BenchBoard docs
official 2026-10-13 leaderboard snapshot · 70 held-out tasks in this world · 4 exploration rounds · 3 answers/task · max reasoning · one matched control trajectory
AlienCode · M4 · hindsight-replay control
Reported by Fudan University / Tencent Hy / Tsinghua University · via BenchBoard docs
official 2026-10-13 leaderboard snapshot · 70 held-out tasks in this world · 4 exploration rounds · 3 answers/task · max reasoning · one matched control trajectory
Show 16 more resultsHide 16 results
AlienCode · M4 · Mean@3 held-out accuracy
Reported by Fudan University / Tencent Hy / Tsinghua University, Tencent Hy / ExplorationBench authors · via BenchBoard docs
official 2026-10-13 leaderboard snapshot · 70 held-out tasks in this world · 4 exploration rounds · 3 answers/task · max reasoning · 3 independent autonomous trajectories
AlienCode · M4 · without-tool control
Reported by Fudan University / Tencent Hy / Tsinghua University · via BenchBoard docs
official 2026-10-13 leaderboard snapshot · 70 held-out tasks in this world · 4 exploration rounds · 3 answers/task · max reasoning · one matched control trajectory
AlienCode · M4 · Worst@3 held-out accuracy
Reported by Fudan University / Tencent Hy / Tsinghua University · via BenchBoard docs
official 2026-10-13 leaderboard snapshot · 70 held-out tasks in this world · 4 exploration rounds · 3 answers/task · max reasoning · 3 independent autonomous trajectories
AlienCode · O@M0 · complete rules before exploration
Reported by Fudan University / Tencent Hy / Tsinghua University · via BenchBoard docs
official 2026-10-13 leaderboard snapshot · 70 held-out tasks in this world · 4 exploration rounds · 3 answers/task · max reasoning · complete-rule reference or auxiliary diagnostic
AlienCode · Rules correctly reported
Reported by Fudan University / Tencent Hy / Tsinghua University · via BenchBoard docs
official 2026-10-13 leaderboard snapshot · 70 held-out tasks in this world · 4 exploration rounds · 3 answers/task · max reasoning · complete-rule reference or auxiliary diagnostic
AlienLogic · A4+O · complete rules after exploration
Reported by Fudan University / Tencent Hy / Tsinghua University · via BenchBoard docs
official 2026-10-13 leaderboard snapshot · 70 held-out tasks in this world · 4 exploration rounds · 3 answers/task · max reasoning · complete-rule reference or auxiliary diagnostic
AlienLogic · Correct refusal on unprovable tasks
Reported by Fudan University / Tencent Hy / Tsinghua University · via BenchBoard docs
official 2026-10-13 leaderboard snapshot · 70 held-out tasks in this world · 4 exploration rounds · 3 answers/task · max reasoning · complete-rule reference or auxiliary diagnostic
AlienLogic · M0 · baseline of Best@3 trajectory
Reported by Fudan University / Tencent Hy / Tsinghua University, Tencent Hy / ExplorationBench authors · via BenchBoard docs
official 2026-10-13 leaderboard snapshot · 70 held-out tasks in this world · 4 exploration rounds · 3 answers/task · max reasoning · 3 independent autonomous trajectories
AlienLogic · M4 · Best@3 held-out accuracy
Reported by Fudan University / Tencent Hy / Tsinghua University, Tencent Hy / ExplorationBench authors · via BenchBoard docs
official 2026-10-13 leaderboard snapshot · 70 held-out tasks in this world · 4 exploration rounds · 3 answers/task · max reasoning · 3 independent autonomous trajectories
AlienLogic · M4 · direct answering control
Reported by Fudan University / Tencent Hy / Tsinghua University · via BenchBoard docs
official 2026-10-13 leaderboard snapshot · 70 held-out tasks in this world · 4 exploration rounds · 3 answers/task · max reasoning · one matched control trajectory
AlienLogic · M4 · fixed-probe control
Reported by Fudan University / Tencent Hy / Tsinghua University · via BenchBoard docs
official 2026-10-13 leaderboard snapshot · 70 held-out tasks in this world · 4 exploration rounds · 3 answers/task · max reasoning · one matched control trajectory
AlienLogic · M4 · hindsight-replay control
Reported by Fudan University / Tencent Hy / Tsinghua University · via BenchBoard docs
official 2026-10-13 leaderboard snapshot · 70 held-out tasks in this world · 4 exploration rounds · 3 answers/task · max reasoning · one matched control trajectory
AlienLogic · M4 · Mean@3 held-out accuracy
Reported by Fudan University / Tencent Hy / Tsinghua University, Tencent Hy / ExplorationBench authors · via BenchBoard docs
official 2026-10-13 leaderboard snapshot · 70 held-out tasks in this world · 4 exploration rounds · 3 answers/task · max reasoning · 3 independent autonomous trajectories
AlienLogic · M4 · without-tool control
Reported by Fudan University / Tencent Hy / Tsinghua University · via BenchBoard docs
official 2026-10-13 leaderboard snapshot · 70 held-out tasks in this world · 4 exploration rounds · 3 answers/task · max reasoning · one matched control trajectory
AlienLogic · M4 · Worst@3 held-out accuracy
Reported by Fudan University / Tencent Hy / Tsinghua University · via BenchBoard docs
official 2026-10-13 leaderboard snapshot · 70 held-out tasks in this world · 4 exploration rounds · 3 answers/task · max reasoning · 3 independent autonomous trajectories
AlienLogic · O@M0 · complete rules before exploration
Reported by Fudan University / Tencent Hy / Tsinghua University · via BenchBoard docs
official 2026-10-13 leaderboard snapshot · 70 held-out tasks in this world · 4 exploration rounds · 3 answers/task · max reasoning · complete-rule reference or auxiliary diagnostic
GraphWalks BFSF1 · Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
HealthBench4 related benchmarks · 13 results
HealthBench
Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
Consensus
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Hard
Source: HealthBench Hard (Arcophos) data docs · Reported by the model's developer. The source relaying it did not verify it.
ChatGPT production (Instant) deployment setting, length-adjusted, GPT-5.6 August Updates PDF column GPT-5.6 Sol (August) (27.1 unadjusted, 1,450 chars)
length-adjusted, max reasoning effort (31.1 unadjusted, 1,751 mean response chars), GPT-5.6 system card Table 6, column GPT-5.6-SOL. API max-effort setting, not the August ChatGPT production setting measured in row 490.
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
Professional
Reported by SpaceXAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
length-adjusted score · rejected requests scored zero · max effort
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI GPT-5.6 launch table
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Opus 5 System Card capability summary · adaptive thinking · max effort unless noted
Source: Clinical Benchmarks data docs · Reported by the model's developer. The source relaying it did not verify it.
Setting not stated
Setting not stated
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
Horangi36 related benchmarks · 36 results
MLE-Bench Lite · HumanRankAtria Dawn Preview official card / report Table 1 · published comparison run · model snapshots and reasoning tiers not specified unless stated · Claude Code · 12-hour task limit · HumanRank score · Reported by Shanghai AI Laboratory · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
MRCR v2 · 256KQwen表:MRCR v2 · 256K · 8-needle · Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
MultiLangBench6 related benchmarks · 6 results
COBOL test pass rate
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI publication · SQLite 3.40.1 from scratch · COBOL · maximum reasoning effort · 272-test recommended subset in public harness; exact publication test selection and API snapshot not stated
20-language average test pass rate
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI publication · SQLite 3.40.1 from scratch · 20-language average · maximum reasoning effort · 272-test recommended subset in public harness; exact publication test selection and API snapshot not stated
Estimated average API cost
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI publication · 20-language average · maximum reasoning effort · estimated API cost per sample · exact API snapshot not stated
Fortran test pass rate
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI publication · SQLite 3.40.1 from scratch · Fortran · maximum reasoning effort · 272-test recommended subset in public harness; exact publication test selection and API snapshot not stated
OCaml test pass rate
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI publication · SQLite 3.40.1 from scratch · OCaml · maximum reasoning effort · 272-test recommended subset in public harness; exact publication test selection and API snapshot not stated
Visual Basic test pass rate
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI publication · SQLite 3.40.1 from scratch · Visual Basic · maximum reasoning effort · 272-test recommended subset in public harness; exact publication test selection and API snapshot not stated
Nejumi 428 related benchmarks · 28 results
NL2Repo-Bench2 results
Reported by DeepSeek · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
DeepSeek V4.1 launch chart · settings unspecified · launch-page table
OpenAI5 related benchmarks · 5 results
Internal Design Tasks
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI internal benchmark
COBOL migration · Exact behavior accuracy
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI publication · COBOL to C++/Java/Kotlin/Python/Rust · hidden byte-for-byte tests · reasoning: low · exact API snapshot and run count not stated
Internal Data Science Tasks
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI internal benchmark
Internal Database Migration Tasks
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI internal benchmark
Internal Research Debugging
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI internal benchmark
PerceptionBench9 related benchmarks · 9 results
Overall accuracy
Reported by Moonshot AI / PerceptionBench authors · via BenchBoard docs
benchmark-owner evaluation · 3,000 verified questions total · open-ended short answers · unified prompts · max reasoning · four-run mean per paper Table 2 · GPT-oss-120B judge
Attribute accuracy
Reported by Moonshot AI / PerceptionBench authors · via BenchBoard docs
benchmark-owner evaluation · 3,000 verified questions total · open-ended short answers · unified prompts · max reasoning · main-table result; run count not specified · GPT-oss-120B judge
Comparison accuracy
Reported by Moonshot AI / PerceptionBench authors · via BenchBoard docs
benchmark-owner evaluation · 3,000 verified questions total · open-ended short answers · unified prompts · max reasoning · main-table result; run count not specified · GPT-oss-120B judge
Contextual integration accuracy
Reported by Moonshot AI / PerceptionBench authors · via BenchBoard docs
benchmark-owner evaluation · 3,000 verified questions total · open-ended short answers · unified prompts · max reasoning · main-table result; run count not specified · GPT-oss-120B judge
Counting accuracy
Reported by Moonshot AI / PerceptionBench authors · via BenchBoard docs
benchmark-owner evaluation · 3,000 verified questions total · open-ended short answers · unified prompts · max reasoning · main-table result; run count not specified · GPT-oss-120B judge
Depth and 3D perception accuracy
Reported by Moonshot AI / PerceptionBench authors · via BenchBoard docs
benchmark-owner evaluation · 3,000 verified questions total · open-ended short answers · unified prompts · max reasoning · main-table result; run count not specified · GPT-oss-120B judge
Show 3 more resultsHide 3 results
Fine-grained recognition accuracy
Reported by Moonshot AI / PerceptionBench authors · via BenchBoard docs
benchmark-owner evaluation · 3,000 verified questions total · open-ended short answers · unified prompts · max reasoning · main-table result; run count not specified · GPT-oss-120B judge
Localization accuracy
Reported by Moonshot AI / PerceptionBench authors · via BenchBoard docs
benchmark-owner evaluation · 3,000 verified questions total · open-ended short answers · unified prompts · max reasoning · main-table result; run count not specified · GPT-oss-120B judge
Visual relation accuracy
Reported by Moonshot AI / PerceptionBench authors · via BenchBoard docs
benchmark-owner evaluation · 3,000 verified questions total · open-ended short answers · unified prompts · max reasoning · main-table result; run count not specified · GPT-oss-120B judge
PLawBenchQwen表:Gemini 3.1 Pro Preview judge · Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
PRBench-FinanceReported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Reported by Z.ai · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Reported by DeepSeek · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
DeepSeek V4.1 launch chart · settings unspecified
QwenQoderBenchQwen内部基准:Claude Code · avg@5 · 6h · 256K · Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
QwenReactBenchQwen内部双语 React 基准 · Claude Code harness · 自动渲染 + 多模态评判 · BT/Elo · Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
RecreationBench paper26 related benchmarks · 26 results
Web · Visual assertion score
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 272k context · 128k max output tokens · Web · nominal 50-task split · platform-specific score
Android · Applications with Prog =100%
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 272k context · 128k max output tokens · Android · nominal 50-task split · platform-specific score
Android · Applications with Prog ≥90%
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 272k context · 128k max output tokens · Android · nominal 50-task split · platform-specific score
Android · Programmatic assertion score
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 272k context · 128k max output tokens · Android · nominal 50-task split · platform-specific score
Android · Visual assertion score
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 272k context · 128k max output tokens · Android · nominal 50-task split · platform-specific score
macOS · Applications with Prog =100%
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 272k context · 128k max output tokens · macOS · nominal 50-task split · platform-specific score
Show 20 more resultsHide 20 results
macOS · Applications with Prog ≥90%
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 272k context · 128k max output tokens · macOS · nominal 50-task split · platform-specific score
macOS · Programmatic assertion score
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 272k context · 128k max output tokens · macOS · nominal 50-task split · platform-specific score
macOS · Visual assertion score
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 272k context · 128k max output tokens · macOS · nominal 50-task split · platform-specific score
Overall · Applications with Prog =100%
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 272k context · 128k max output tokens · Overall · 250 held-out tasks · equal-weight platform macro average
Overall · Applications with Prog ≥90%
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 272k context · 128k max output tokens · Overall · 250 held-out tasks · equal-weight platform macro average
Overall · Estimated API cost per task
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 272k context · 128k max output tokens · estimated mean API cost per task · 90% cache reads assumed
Overall · Programmatic + visual mean
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 272k context · 128k max output tokens · Overall · 250 held-out tasks · equal-weight platform macro average
Overall · Programmatic assertion score
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 272k context · 128k max output tokens · Overall · 250 held-out tasks · equal-weight platform macro average
Overall · Visual assertion score
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 272k context · 128k max output tokens · Overall · 250 held-out tasks · equal-weight platform macro average
Ubuntu · Applications with Prog =100%
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 272k context · 128k max output tokens · Ubuntu · nominal 50-task split · platform-specific score
Ubuntu · Applications with Prog ≥90%
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 272k context · 128k max output tokens · Ubuntu · nominal 50-task split · platform-specific score
Ubuntu · Programmatic assertion score
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 272k context · 128k max output tokens · Ubuntu · nominal 50-task split · platform-specific score
Ubuntu · Visual assertion score
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 272k context · 128k max output tokens · Ubuntu · nominal 50-task split · platform-specific score
Web · Applications with Prog =100%
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 272k context · 128k max output tokens · Web · nominal 50-task split · platform-specific score
Web · Applications with Prog ≥90%
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 272k context · 128k max output tokens · Web · nominal 50-task split · platform-specific score
Web · Programmatic assertion score
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 272k context · 128k max output tokens · Web · nominal 50-task split · platform-specific score
Windows · Applications with Prog =100%
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 272k context · 128k max output tokens · Windows · nominal 50-task split · platform-specific score
Windows · Applications with Prog ≥90%
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 272k context · 128k max output tokens · Windows · nominal 50-task split · platform-specific score
Windows · Programmatic assertion score
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 272k context · 128k max output tokens · Windows · nominal 50-task split · platform-specific score
Windows · Visual assertion score
Reported by QwenLM / RecreationWorld authors · via BenchBoard docs
RecreationWorld benchmark-owner paper run · 250 held-out tasks · GUI exploration + coding tools + visual verification · max reasoning effort per site data · 272k context · 128k max output tokens · Windows · nominal 50-task split · platform-specific score
SkillsBench9 related benchmarks · 9 results
1.1 · With Skills
Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Qwen他测:v1.1 · 87 tasks · 每题3次均值;Opus/Fable=Claude Code,GPT=Codex,Qwen=OpenCode
Text-only (79 tasks)
Reported by Tencent Hy · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Hy4 official Benchmark Appendix · highest available reasoning setting · Claude Code · 79 self-contained text-only tasks · three-run average
Text-only (79 tasks) · OpenHands
Reported by Shanghai AI Laboratory · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Atria Table 1 / Appendix B · OpenHands within AgentCompass · 79 self-contained text-only tasks · three-run average · reasoning tiers and task-source commit not stated
Vals · With Skills · Accuracy
Reported by Vals AI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Vals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · With Skills · system=openai/gpt-5.6-sol · provider=OpenAI · reasoning_effort=max · max_output_tokens=128000
Vals · With Skills · Mean cost / test
Reported by Vals AI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Vals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · With Skills · system=openai/gpt-5.6-sol · provider=OpenAI · reasoning_effort=max · max_output_tokens=128000
Vals · With Skills · Mean duration / test
Reported by Vals AI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Vals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · With Skills · system=openai/gpt-5.6-sol · provider=OpenAI · reasoning_effort=max · max_output_tokens=128000
Show 3 more resultsHide 3 results
Vals · Without Skills · Accuracy
Reported by Vals AI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Vals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · Without Skills · system=openai/gpt-5.6-sol · provider=OpenAI · reasoning_effort=max · max_output_tokens=128000
Vals · Without Skills · Mean cost / test
Reported by Vals AI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Vals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · Without Skills · system=openai/gpt-5.6-sol · provider=OpenAI · reasoning_effort=max · max_output_tokens=128000
Vals · Without Skills · Mean duration / test
Reported by Vals AI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Vals AI · updated 2026-09-27 · metadata version 1 · 87 tasks in trace charts · 3 trials/task/condition · per-task mean then task average · OpenHands-CLI@ · Without Skills · system=openai/gpt-5.6-sol · provider=OpenAI · reasoning_effort=max · max_output_tokens=128000
SWE-Marathon v1.1native provider harnesses · 8 trials/task · resolution rate · max effort · Reported by SpaceXAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Vals16 related benchmarks · 18 results
Show 12 more resultsHide 12 results
MedCode
Source: Clinical Benchmarks data docs · Sep 2026
Setting not stated
Source: Vals AI
Setting not stated
MedScribe
Source: Clinical Benchmarks data docs · Sep 2026
Setting not stated
Source: Vals AI
Setting not stated
MortgageTax
Source: Vals AI
WideSearch · Atria paper scoreAtria Dawn Preview official card / report Table 1 · published comparison run · model snapshots and reasoning tiers not specified unless stated · in-house search/visit/python harness · 500 steps · 256K context / 64K max output · score subtype unstated · Reported by Shanghai AI Laboratory · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Show 12 more resultsHide 12 results
Source: OpenRouter benchmark exports · Aug 2026
exa search · OpenRouter's search tool · up to 1 turn · high effort
exa search · OpenRouter's search tool · up to 25 turns · high effort
exa search · OpenRouter's search tool · up to 5 turns · high effort
the model's own search · OpenRouter's search tool · up to 1 turn · high effort
the model's own search · OpenRouter's search tool · up to 25 turns · high effort
the model's own search · OpenRouter's search tool · up to 5 turns · high effort
parallel search · OpenRouter's search tool · up to 1 turn · high effort
parallel search · OpenRouter's search tool · up to 25 turns · high effort
parallel search · OpenRouter's search tool · up to 5 turns · high effort
perplexity search · OpenRouter's search tool · up to 1 turn · high effort
perplexity search · OpenRouter's search tool · up to 25 turns · high effort
perplexity search · OpenRouter's search tool · up to 5 turns · high effort
Workspace-Bench3 related benchmarks · 3 results
1.0 · Full
Reported by Qwen · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Qwen3.8 官方模型卡对照表 · Workspace-Bench 1.0
English Full · Atria run
Reported by Shanghai AI Laboratory · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Atria Dawn Preview official card / report Table 1 · published comparison run · model snapshots and reasoning tiers not specified unless stated · AgentCompass · English Full · two-hour limit · 2 CPUs / 8 GiB · Seed 2.0 Lite judge · GPT in Codex, others in Claude Code · metric subtype unstated
English Lite · Atria run
Reported by Shanghai AI Laboratory · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Atria Dawn Preview official card / report Table 1 · published comparison run · model snapshots and reasoning tiers not specified unless stated · AgentCompass · English Lite · two-hour limit · 2 CPUs / 8 GiB · Seed 2.0 Lite judge · GPT in Codex, others in Claude Code · metric subtype unstated
ZeroBench · MainDeepSeek V4.1 launch chart · settings unspecified · with tools · Reported by DeepSeek · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
τ³-Bench · Banking Knowledge · Atria runAtria Dawn Preview official card / report Table 1 · published comparison run · model snapshots and reasoning tiers not specified unless stated · 97 Banking Knowledge tasks · modified Tau2/Tau3 runner from commit a2c0247 · alltools BM25/dense/shell retrieval · GPT-5.2 low judge and simulator · 200-step limit · Reported by Shanghai AI Laboratory · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.

Data credits: BenchBoard by SYuan03 (github.com/SYuan03/benchboard). MIT covers its code and data organization; no results licence is inferred. Numeric relays retain the original stating-source links. · Data by LLM Stats · Epoch AI Benchmarking Hub, CC BY 4.0 (licence) · Clinical Benchmarks (clinicalbenchmarks.ai), CC BY 4.0; fractions converted to percentages; source documents retain their own terms (licence) · LMArena, CC BY 4.0 (licence) · Baromètre de l’écriture IA, Johannes Eva; public numeric ranking data, © 2026 Johannes Eva, no results licence stated · OpenRouter public benchmark leaderboard and search-lane exports, CC BY 4.0. Fractional source scores converted to percentages; immutable dated snapshots. (licence) · BullshitBench by Peter Gostev, MIT (licence) · EQ-Bench (Samuel J. Paech) — cite paech2023eqbench · Deep20Bench edition 1.1, Patrick Heusser. Results CC BY 4.0; native question scores and fraction-to-percent success rates. (licence) · Keelemudelite mõõdupuu: Independent LLM leaderboard for Estonian. Eesti Keele Instituut. https://moodupuu.eki.ee, CC BY 4.0. Latest per-benchmark results selected with partial-run flags retained. (licence) · ExploitGym leaderboard (Wang et al., UC Berkeley), arXiv:2605.11086. No licence is stated for the result data; the repository's Apache-2.0 licence covers its code only, and task data keeps its upstream licences. Literal exploit counts; hidden rows omitted. · Arcophos. HealthBench Hard Leaderboard. https://healthbenchhard.ai. CC BY 4.0. Source document linked on each result. (licence) · Horangi 4, Weights & Biases Korea; public publisher-selected numeric evaluation results · LLM Chess by Maxim Saplin, Apache-2.0 (Copyright [2024] [Maxim Saplin]) (licence) · MathArena (ETH SRI / INSAIT), MIT — cite dekoninck2026matharena (licence) · MCPMark (eval-sys), mcpmark.ai Verified leaderboard. Single-run Pass@1 fractions shown as percentages. · MineBench, MIT (licence) · Nejumi Leaderboard 4, Weights & Biases Japan; public publisher-selected numeric evaluation results · Outback Yak Research, NepNewsBench v2 research draft. Public numerical aggregates from a candidate preview that the publisher says is not evidence of publication; no release licence assigned and none asserted here. · Steel.dev Agent Leaderboards, MIT (licence) · Trishul Lab, PutnamBench public submitted results. Model identities and scaffold budgets preserved; result licence unstated. · Hugging Face official benchmark leaderboards; licence varies by benchmark · SnakeBench (Greg Kamradt / ARC Prize Foundation), MIT (licence) · SWE-bench-Live (Microsoft and contributors), swe-bench-live.github.io. Resolve rates computed from the published report counts. · Sierra Research tau2-bench leaderboard submissions (github.com/sierra-research/tau2-bench), MIT (licence) · Vectara Hallucination Leaderboard, Apache-2.0 (licence) · WeirdML v3 by Håvard Tveit Ihle (Norwegian Defence Research Establishment). API costs supported by Epoch AI, METR and NDRE.

Around the web

Also known as: aws-mantle/gpt-5.6-sol, azure/eu/gpt-5.6-sol, azure/gpt-5.6-sol, azure/gpt-5.6-sol-2026-07-09, azure/gpt-5.6-sol@eastus2, azure/gpt-5.6-sol@swedencentral, azure/us-gov/gpt-5.6-sol, azure/us/gpt-5.6-sol, azure_ai/gpt-5.6-sol, azure_ai/gpt-5.6-sol-2026-07-09, bedrock_mantle/openai.gpt-5.6-sol, chatgpt/gpt-5.6-sol, databricks-gpt-5-6-sol, databricks/databricks-gpt-5-6-sol, github_copilot/gpt-5.6-sol, global.openai.gpt-5.6-sol, gmi/openai/gpt-5.6-sol, gpt-5-6-sol, gpt-5.6 sol, gpt-5.6 sol pro, gpt-5.6-sol-2026-07-09, gpt-5.6-sol-pro, gpt-5.6-sol-xhigh, gpt-5.6-sol-xhigh-codex-harness, gpt-5.6-sol:web, gpt-5.6-sol@eu, gpt-56-sol, openai-gpt-5.6-sol, openai-gpt-5.6-sol-pro, openai-gpt-56-sol, openai.gpt-5.6-sol, openai/gpt-5.6-sol, openai/gpt-5.6-sol-batch, openai/gpt-5.6-sol-fast, openai/gpt-5.6-sol-pro, openai/gpt-5.6-sol-pro:batch, openai/gpt-5.6-sol:batch, perplexity/openai/gpt-5.6-sol, replicate/openai/gpt-5.6-sol, us.openai.gpt-5.6-sol

Like this one. Tap or hover any dotted value, label or icon to see where it comes from and how it is measured. open details when tapped or hovered. How this data works