Skip to content

Google · Chat

Gemini 3.1 Pro Preview

Google 🇺🇸ChatGemini familyOne of its sources says the weights aren't publicly available.
From Modelrunner AI's listing:
Solve hard reasoning, coding and agentic tasks from a text prompt with a 1M-token-context thinking model that Google reports at 77.1% on ARC-AGI-2.
llminfo IQ
136.0
Maker's price
$4.50 blended / 1M
Max context
1M
Listings
92

Evidence and sources

llminfo IQ 136.0, ranked among chat models. From 595 benchmarks. Plenty of data behind it.How llminfo IQ is computedCoding IQ 110.0, 95% interval 102–119. From 27 benchmark families across 17 sources. Plenty of data behind it. Uses the model's best published results. It has its own scale, separate from llminfo IQ.How Coding IQ is computedFair value (49.8). Costs per task about what its benchmark results predict. From 89 benchmarks with measured task costs. Plenty of data behind it.How Value for money worksYap score: tokens used per coding task vs. models with the same resultsTerse (38 ±12). Generates fewer tokens than models with similar coding results. Measured at default effort. A fair amount of data behind it. By effort: default 38 · low 47 · high 41.Median 14,198 output tokens per task at default effort.How the Yap score worksReleased 19 Feb 2026Maker HQ: US (according to Google's own website)
All rankings: #25 in Google Trends interest this week #96 on OpenRouter this week #98 on OpenRouter this month #100 on OpenRouter today #163 on NanoGPT recentlyAll rankings: #25 in Google Trends interest this week #96 on OpenRouter this week #98 on OpenRouter this month #100 on OpenRouter today #163 on NanoGPT recentlySources: Google Generative AI API data docs, Writer Pricing, ModelRunner.ai data, AI/ML API data, llminfo researchPrimary data from Google Generative AI API from Writer Pricing: variant from ModelRunner.ai: weights availability, description from AI/ML API: primary mode from llminfo research: release date

Advantages and trade-offs

Compared with 10 chat models, picked mainly by popularity and IQ

Advantages

Trade-offs

No open weights

The weights aren't public, so you can't run it yourself.

We only make a claim when there's enough data on models of the same type. Updated 22 Nov, 06:07 UTC.Prices come from the model's maker where they sell it, otherwise their official partner, otherwise the market median. This comparison uses 10 maker's prices.

Input & output

as stated by the model's providers
Text
Images
Audio
Video
Files
Text
Images

Capabilities

Supported operations and provider-stated features

VisionReasoningFunction callingStructured outputPrompt cachingPDF inputWeb searchAudio inputBatch API

Listings

 Googlemodels/gemini-3.1-pro-previewSources: GenAI Prices data, Google Generative AI API data docs, LiteLLM data, Models.dev data, Portkey Models data1M$2.00>128k $4.00>200k $4.00cache read $0.20batch $1.00$12.00>128k $18.00>200k $18.00batch $6.00——$0.00012———
 Nous Portalgoogle/gemini-3.1-pro-preview:batch · batchSource: Nous Portal Inference Gateway data1M$1.00$6.00$1.00——$0.000001——
 OpenRouterInference by Googlegoogle/gemini-3.1-pro-preview · global/flexSource: OpenRouter data docs1M$1.00cache read $0.10$6.00$1.00$0.10—$0.0000013592,410ms
 RequestyInference by Googlevertex/gemini-3.1-pro-preview:flex · flexSource: Requesty data1M$1.00>200k $2.00cache read $0.10$6.00>200k $9.00——————
 Vercelgoogle/gemini-3.1-pro-preview-batch · batchSource: Vercel AI Gateway data1M$1.00cache read $0.20$6.00——————
 Kenarigemini-3-1-proSource: Models.dev data1M$1.20cache read $0.12$7.14——————
 Tokensmindgemini-3.1-pro-previewSource: TokensMindNot published by the provider$1.40$8.40——————
 Meganovagemini/gemini-3.1-pro-previewSource: MegaNova data1M$1.60$9.60——————
 Tokenmixgemini-3.1-proSources: TokenMix data, TokenMix PricingNot published by the provider$1.94$11.64——————
 302aigemini-3.1-pro-previewSource: Models.dev data1M$2.00$12.00——————

13 listings without a price are hidden.

Benchmarks

Named effort levels run from lowest to highest. Select a result to see its source details.

Agentic

Arena AgentSource: LMArena data
BrowseComp5 results
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Fable/Mythos 5 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted
Opus 4.8 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI GPT-5.6 launch table
Reported by Google DeepMind · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Search + Python + Browse
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
Design Arena EloSource: Design Arena Leaderboards data · Nov 2026
Source: LLM Stats data
Setting not stated
Reported by Google DeepMind · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Gemini 3.5 Flash Model Card · May 2026
Reported by Meta, Meta Superintelligence Labs · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
high · Vals AI result
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Opus 4.8 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted
GDPval-AA3 related benchmarks · 8 results
GDPval-AA
Reported by Google DeepMind · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Setting not stated
Gemini 3.5 Flash Model Card · May 2026
Published Score
Reported by MiniMax · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
MiniMax M2.7 official benchmark overview · exact harness/version not stated
v2
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Fable/Mythos 5 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted
Opus 4.8 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted
Reported by Google DeepMind · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Gemini 3.6 Flash Model Card · July 2026
Reported by Meta · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
high · Artificial Analysis leaderboard
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI GPT-5.6 launch table
MCP Atlas5 results
Reported by Google DeepMind · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Gemini 3.5 Flash Model Card · May 2026
Reported by Meta, Meta Superintelligence Labs · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
high · Scale AI MCP Atlas
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Opus 4.8 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
MLE-BenchGemini 3.6 Flash Model Card · July 2026 · Reported by Google DeepMind · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
MLE-Bench LiteMiniMax M2.7 official benchmark overview · exact harness/version not stated · Reported by MiniMax · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenHandsSource: OpenHands Index data
OSWorld-Verified3 related benchmarks · 7 results
OSWorld-Verified
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Fable/Mythos 5 System Card setting
Opus 4.8 System Card setting
Reported by Google DeepMind · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Gemini 3.5 Flash Model Card · May 2026
Gemini 3.6 Flash Model Card · July 2026
Reported by Meta, Meta Superintelligence Labs · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
high · 361-task split excluding Google Drive · GUI-only · 1920×1080 · 200 steps · pass@1
OSWorld 2.0 · Binary
Reported by Meta · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
high · 108 tasks · GUI-only · no exec · 500 steps · binary completion
OSWorld 2.0 · Partial
Reported by Meta · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
high · 108 tasks · GUI-only · no exec · 500 steps · partial reward
SWE-Bench Pro9 results
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Fable/Mythos 5 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted
Opus 4.8 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted
Reported by Google DeepMind · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Gemini 3.5 Flash Model Card · May 2026 · Public · single attempt
Gemini 3.6 Flash Model Card · July 2026 · Public
public subset · single attempt
Reported by Meta, Meta Superintelligence Labs · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
high · Scale AI · mini-swe-agent for Muse Spark 1.1
Show 3 more resultsHide 3 results
Reported by MiniMax · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
MiniMax M2.7 official benchmark overview · exact harness/version not stated
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI GPT-5.6 launch table
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
Source: OpenRouter benchmark exports · Oct 2026
19 Feb 2026 version · 90-day results
Across 2 providers lowest 74% median 75% highest 76%OpenRouter's automatic routing 77%
Show results from 2 providers and OpenRouter's automatic routingHide results from 2 providers and OpenRouter's automatic routing
  • OpenRouter's automatic routing
  • Google
  • Google AI Studio
Terminal-Bench4 related benchmarks · 9 results
2.0 · Terminus-2
Reported by Google DeepMind · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Terminal-Bench 2.0
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Terminal-Bench 2.1
Reported by Google DeepMind · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Gemini 3.5 Flash Model Card · May 2026 · Terminus-2
Gemini 3.6 Flash Model Card · July 2026 · Terminus-2
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Opus 4.8 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted
Reported by Meta, Meta Superintelligence Labs · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
high · 89 tasks · bash-only · pass@1 · five-attempt mean
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI GPT-5.6 launch table
WebArena-Verifiedhigh · OpenClaw · DOM + screenshots · full 812 tasks · 150 tool calls · pass@1 · Reported by Meta · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.

Human preference

Arena ELO68 related benchmarks · 68 results
Source: LMArena data · Nov 2026
Arena ELO
Chinese
Show 62 more resultsHide 62 results
Design Arena EloSource: Design Arena Leaderboards data · Nov 2026

Reasoning

ARC-AGI4 related benchmarks · 6 results
ARC-AGI 2
Source: Epoch AI Benchmarking Hub data
Setting not stated
Reported by Google DeepMind · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Gemini 3.5 Flash Model Card · May 2026
ARC-AGI-3
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI GPT-5.6 launch table
v2
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Fable/Mythos 5 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted
Opus 4.8 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Fable/Mythos 5 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted
Reported by Google DeepMind · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Gemini 3.5 Flash Model Card · May 2026 · normalized score
CL-bench9 related benchmarks · 22 results
Overall
Reported by Tencent Hunyuan / Fudan NLP Group · via BenchBoard docs
benchmark-owner leaderboard run · site model-mode label: Gemini 3.1 Pro (high) · 1899 tasks · GPT-5.1 judge · low reasoning effort · single run; no standard deviation · candidate API snapshot and exact candidate harness not specified on public leaderboard
Source: Epoch AI Benchmarking Hub data
Setting not stated
Domain Knowledge Reasoning
Reported by Tencent Hunyuan / Fudan NLP Group · via BenchBoard docs
benchmark-owner leaderboard run · site model-mode label: Gemini 3.1 Pro (high) · 1899 tasks · GPT-5.1 judge · low reasoning effort · single run; no standard deviation · candidate API snapshot and exact candidate harness not specified on public leaderboard
Source: Epoch AI Benchmarking Hub data
Setting not stated
Empirical Discovery & Simulation
Reported by Tencent Hunyuan / Fudan NLP Group · via BenchBoard docs
benchmark-owner leaderboard run · site model-mode label: Gemini 3.1 Pro (high) · 1899 tasks · GPT-5.1 judge · low reasoning effort · single run; no standard deviation · candidate API snapshot and exact candidate harness not specified on public leaderboard
Source: Epoch AI Benchmarking Hub data
Setting not stated
Life · Behavioral Records & Activity Trails
Reported by Tencent Hunyuan / Fudan NLP Group · via BenchBoard docs
benchmark-owner leaderboard run · site model-mode label: Gemini 3.1 Pro · 405 tasks · GPT-5.1 judge · high reasoning effort · single run; no standard deviation · candidate API snapshot and exact candidate harness not specified on public leaderboard
benchmark-owner leaderboard run · site model-mode label: Gemini 3.1 Pro (High) · 405 tasks · GPT-5.1 judge · high reasoning effort · candidate API snapshot and exact candidate harness not specified on public leaderboard
Source: Epoch AI Benchmarking Hub data
Setting not stated
Life · Communication & Social Interactions
Reported by Tencent Hunyuan / Fudan NLP Group · via BenchBoard docs
benchmark-owner leaderboard run · site model-mode label: Gemini 3.1 Pro · 405 tasks · GPT-5.1 judge · high reasoning effort · single run; no standard deviation · candidate API snapshot and exact candidate harness not specified on public leaderboard
benchmark-owner leaderboard run · site model-mode label: Gemini 3.1 Pro (High) · 405 tasks · GPT-5.1 judge · high reasoning effort · candidate API snapshot and exact candidate harness not specified on public leaderboard
Source: Epoch AI Benchmarking Hub data
Setting not stated
Life · Fragmented Information & Revisions
Reported by Tencent Hunyuan / Fudan NLP Group · via BenchBoard docs
benchmark-owner leaderboard run · site model-mode label: Gemini 3.1 Pro · 405 tasks · GPT-5.1 judge · high reasoning effort · single run; no standard deviation · candidate API snapshot and exact candidate harness not specified on public leaderboard
benchmark-owner leaderboard run · site model-mode label: Gemini 3.1 Pro (High) · 405 tasks · GPT-5.1 judge · high reasoning effort · candidate API snapshot and exact candidate harness not specified on public leaderboard
Source: Epoch AI Benchmarking Hub data
Setting not stated
Show 7 more resultsHide 7 results
Life · Overall
Reported by Tencent Hunyuan / Fudan NLP Group · via BenchBoard docs
benchmark-owner leaderboard run · site model-mode label: Gemini 3.1 Pro · 405 tasks · GPT-5.1 judge · high reasoning effort · single run; no standard deviation · candidate API snapshot and exact candidate harness not specified on public leaderboard
benchmark-owner leaderboard run · site model-mode label: Gemini 3.1 Pro (High) · 405 tasks · GPT-5.1 judge · high reasoning effort · candidate API snapshot and exact candidate harness not specified on public leaderboard
Source: Epoch AI Benchmarking Hub data
Setting not stated
Procedural Task Execution
Reported by Tencent Hunyuan / Fudan NLP Group · via BenchBoard docs
benchmark-owner leaderboard run · site model-mode label: Gemini 3.1 Pro (high) · 1899 tasks · GPT-5.1 judge · low reasoning effort · single run; no standard deviation · candidate API snapshot and exact candidate harness not specified on public leaderboard
Source: Epoch AI Benchmarking Hub data
Setting not stated
Rule System Application
Reported by Tencent Hunyuan / Fudan NLP Group · via BenchBoard docs
benchmark-owner leaderboard run · site model-mode label: Gemini 3.1 Pro (high) · 1899 tasks · GPT-5.1 judge · low reasoning effort · single run; no standard deviation · candidate API snapshot and exact candidate harness not specified on public leaderboard
Source: Epoch AI Benchmarking Hub data
Setting not stated
DeepSearchQAhigh · browser search/open/find · gpt-oss-120b answer extraction · Reported by Meta · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
GDP.pdf2 results
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Fable/Mythos 5 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI GPT-5.6 launch table
GeneBench ProOpenAI GPT-5.6 launch table · Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Fable/Mythos 5 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted · no tools
Opus 4.8 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted · no tools
Reported by Google DeepMind · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Gemini 3.5 Flash Model Card · May 2026 · full text + multimodal set
Reported by Meta · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
high · no tools · full 2,500 questions · o3-mini judge
high · no tools · self-reported value quoted beside Meta's 45.4 reproduction
Show 1 more resultHide 1 result
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Fable/Mythos 5 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted · with tools
Opus 4.8 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted · with tools
Reported by Meta, Meta Superintelligence Labs · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
high · full 2,500 questions · bash + browser tools · o3-mini judge
Reported by Google DeepMind · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
search with blocklist + code
OfficeQA ProFable/Mythos 5 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted · Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
t2-benchSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Reported by Meta, Meta Superintelligence Labs · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
high · Toolathlon-Verified · official harness
Reported by MiniMax · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
MiniMax M2.7 official benchmark overview · exact harness/version not stated
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI GPT-5.6 launch table

Multilingual

Bangla LLM BenchSource: Bangla LLM Bench data docs
MMMLU2 results
Reported by Google DeepMind · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Setting not stated
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
NepNewsBench v2 draft40 related benchmarks · 40 results
entity F1 (all questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
Google AI Studio via openrouter, reasoning low
core fact recall (all questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
Google AI Studio via openrouter, reasoning low
core fact recall (English-source questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
Google AI Studio via openrouter, reasoning low
core fact recall (mixed English and Nepali source questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
Google AI Studio via openrouter, reasoning low
core fact recall (Nepali-source questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
Google AI Studio via openrouter, reasoning low
coverage precision proxy (all questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
Google AI Studio via openrouter, reasoning low
Show 34 more resultsHide 34 results
coverage precision proxy (English-source questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
Google AI Studio via openrouter, reasoning low
coverage precision proxy (Nepali-source questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
Google AI Studio via openrouter, reasoning low
effective score (all questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
Google AI Studio via openrouter, reasoning low
effective score (English-source questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
Google AI Studio via openrouter, reasoning low
effective score (mixed English and Nepali source questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
Google AI Studio via openrouter, reasoning low
effective score (Nepali-source questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
Google AI Studio via openrouter, reasoning low
English prose quality (all questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
Google AI Studio via openrouter, reasoning low
English prose quality (English-source questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
Google AI Studio via openrouter, reasoning low
English prose quality (Nepali-source questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
Google AI Studio via openrouter, reasoning low
entity F1 (English-source questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
Google AI Studio via openrouter, reasoning low
entity F1 (mixed English and Nepali source questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
Google AI Studio via openrouter, reasoning low
entity F1 (Nepali-source questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
Google AI Studio via openrouter, reasoning low
entity precision (all questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
Google AI Studio via openrouter, reasoning low
entity precision (English-source questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
Google AI Studio via openrouter, reasoning low
entity precision (mixed English and Nepali source questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
Google AI Studio via openrouter, reasoning low
entity precision (Nepali-source questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
Google AI Studio via openrouter, reasoning low
entity recall (all questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
Google AI Studio via openrouter, reasoning low
entity recall (English-source questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
Google AI Studio via openrouter, reasoning low
entity recall (mixed English and Nepali source questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
Google AI Studio via openrouter, reasoning low
entity recall (Nepali-source questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
Google AI Studio via openrouter, reasoning low
Nepali prose quality (all questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
Google AI Studio via openrouter, reasoning low
Nepali prose quality (English-source questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
Google AI Studio via openrouter, reasoning low
Nepali prose quality (Nepali-source questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
Google AI Studio via openrouter, reasoning low
supporting fact recall (all questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
Google AI Studio via openrouter, reasoning low
supporting fact recall (English-source questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
Google AI Studio via openrouter, reasoning low
supporting fact recall (Nepali-source questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
Google AI Studio via openrouter, reasoning low
valid-answer quality (all questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
Google AI Studio via openrouter, reasoning low
valid-answer quality (English-source questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
Google AI Studio via openrouter, reasoning low
valid-answer quality (Nepali-source questions)
Source: NepNewsBench v2 (draft candidate release) data · Sep 2026
Google AI Studio via openrouter, reasoning low

General

Coding

Arena ELOSource: LMArena data · Nov 2026
SciCode2 results
Reported by Google DeepMind · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Setting not stated
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
SWE-bench Verified6 related benchmarks · 9 results
SWE-bench Verified
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Fable/Mythos 5 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted
Opus 4.8 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted
Reported by Google DeepMind · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated
(Epoch)
Source: Epoch AI Benchmarking Hub data
SWE-bench-Live MultiLang Go
Source: SWE-bench-Live leaderboard data · Jul 2026
scaffold: ami-agent, set: go, instances: 138
ValsSource: Vals AI
VIBE-ProMiniMax M2.7 official benchmark overview · exact harness/version not stated · Reported by MiniMax · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.

Long context

Reported by Google DeepMind · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
128K cumulative average
Gemini 3.5 Flash Model Card · May 2026 · 128K cumulative average
Gemini 3.6 Flash Model Card · July 2026 · 128K cumulative average

OCR

OmniDocBenchSource: Nanonets IDP data

Science

GPQASource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
GPQA Diamond8 results
Source: Epoch AI Benchmarking Hub data
Reasoning effort
Effort not reported
Reported by Google DeepMind · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI GPT-5.6 launch table
Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Opus 4.8 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted
Source: OpenRouter benchmark exports · Oct 2026
19 Feb 2026 version · 90-day results
Across 2 providers lowest 95.1% median 95.2% highest 95.2%OpenRouter's automatic routing 94.5%
Show results from 2 providers and OpenRouter's automatic routingHide results from 2 providers and OpenRouter's automatic routing
  • Google
  • Google AI Studio
  • OpenRouter's automatic routing
ValsSource: Vals AI

Math

ArxivMathFable/Mythos 5 System Card capability summary · adaptive thinking · max effort · 5 trials unless noted · Reported by Anthropic · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
MathVisionSenseNova 6.7 Flash-Lite official benchmark chart · fixed boxed-answer prompt · peers use the higher reported boxed/unboxed run · Reported by SenseTime / OpenSenseNova · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.

Vision

Reported by Google DeepMind · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Gemini 3.6 Flash Model Card · July 2026 · reasoning
Gemini 3.5 Flash Model Card · May 2026 · reasoning · no tools
Reported by Meta, Meta Superintelligence Labs · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
high · with code execution · 1,000 validation questions · gpt-oss-120b high judge
MMMU-Pro4 results
Reported by Google DeepMind · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Gemini 3.5 Flash Model Card · May 2026 · no tools
no tools · average of Standard (10 options) and Vision
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Setting not stated

Speech

Factuality

Safety

PhareSource: Phare

Instruction following

Knowledge

Other

ArmBench-LLM Armenian22 related benchmarks · 22 results
Artificial Analysis3 related benchmarks · 3 results
Intelligence Index
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Artificial Analysis Intelligence Index v4.1
Version Unspecified
Reported by MiniMax · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
MiniMax M2.7 official benchmark overview · exact harness/version not stated
Claw-Eval v1.1Reported by Claw-Eval · via BenchBoard docs
DeepSWE v1.13 results
Reported by Google DeepMind · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Gemini 3.6 Flash Model Card · July 2026
Reported by Meta, Meta Superintelligence Labs · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
high · mini-swe-agent fork · no internet · 64GB VM · pass@1 · five-attempt mean
Reported by OpenAI · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OpenAI GPT-5.6 launch table
E-Commerce Bench7 related benchmarks · 7 results
Mean final assets
Reported by E-Commerce Bench authors / Alibaba Qwen · via BenchBoard docs
benchmark-owner 365-day merchant simulation · five independent episodes · up to four stores · initial balance ¥100,000 · thinking enabled where supported; effort not specified · exact API snapshot not stated for unversioned labels
AnchorRatio · repeat-order pricing
Reported by E-Commerce Bench authors / Alibaba Qwen · via BenchBoard docs
benchmark-owner 365-day merchant simulation · five independent episodes · up to four stores · initial balance ¥100,000 · thinking enabled where supported; effort not specified · exact API snapshot not stated for unversioned labels
BadSpend · fraudulent-supplier spend
Reported by E-Commerce Bench authors / Alibaba Qwen · via BenchBoard docs
benchmark-owner 365-day merchant simulation · five independent episodes · up to four stores · initial balance ¥100,000 · thinking enabled where supported; effort not specified · exact API snapshot not stated for unversioned labels
Controllable return
Reported by E-Commerce Bench authors / Alibaba Qwen · via BenchBoard docs
benchmark-owner 365-day merchant simulation · five independent episodes · up to four stores · initial balance ¥100,000 · thinking enabled where supported; effort not specified · exact API snapshot not stated for unversioned labels
CSE+ · honest-deal negotiation
Reported by E-Commerce Bench authors / Alibaba Qwen · via BenchBoard docs
benchmark-owner 365-day merchant simulation · five independent episodes · up to four stores · initial balance ¥100,000 · thinking enabled where supported; effort not specified · exact API snapshot not stated for unversioned labels
Drawdown / peak assets
Reported by E-Commerce Bench authors / Alibaba Qwen · via BenchBoard docs
benchmark-owner 365-day merchant simulation · five independent episodes · up to four stores · initial balance ¥100,000 · thinking enabled where supported; effort not specified · exact API snapshot not stated for unversioned labels
Show 1 more resultHide 1 result
Yuan per tool call
Reported by E-Commerce Bench authors / Alibaba Qwen · via BenchBoard docs
benchmark-owner 365-day merchant simulation · five independent episodes · up to four stores · initial balance ¥100,000 · thinking enabled where supported; effort not specified · exact API snapshot not stated for unversioned labels
FrontierCSReported by ByteDance Seed · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
HealthBench Professionalhigh · 525 items · length-normalized · GPT-5.4 low grader · Reported by Meta · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Horangi36 related benchmarks · 36 results
4: ALT_환각방지
Source: Horangi Korean Leaderboard 4 data docs
gemini-3.1-pro-preview: high-effort
4 subset: haerae_bench_v1_rc
Source: Horangi Korean Leaderboard 4 data docs
gemini-3.1-pro-preview: high-effort
4 subset: haerae_bench_v1_wo_rc
Source: Horangi Korean Leaderboard 4 data docs
gemini-3.1-pro-preview: high-effort
4 subset: hrm8k
Source: Horangi Korean Leaderboard 4 data docs
gemini-3.1-pro-preview: high-effort
4 subset: kmmlu
Source: Horangi Korean Leaderboard 4 data docs
gemini-3.1-pro-preview: high-effort
4 subset: kmmlu_pro
Source: Horangi Korean Leaderboard 4 data docs
gemini-3.1-pro-preview: high-effort
Show 30 more resultsHide 30 results
4 subset: ko_aime2025
Source: Horangi Korean Leaderboard 4 data docs
gemini-3.1-pro-preview: high-effort
4 subset: ko_arc_agi
Source: Horangi Korean Leaderboard 4 data docs
gemini-3.1-pro-preview: high-effort
4 subset: ko_balt_700_semantic
Source: Horangi Korean Leaderboard 4 data docs
gemini-3.1-pro-preview: high-effort
4 subset: ko_balt_700_syntax
Source: Horangi Korean Leaderboard 4 data docs
gemini-3.1-pro-preview: high-effort
4 subset: ko_hallulens_nonexistent
Source: Horangi Korean Leaderboard 4 data docs
gemini-3.1-pro-preview: high-effort
4 subset: ko_hallulens_wikiqa
Source: Horangi Korean Leaderboard 4 data docs
gemini-3.1-pro-preview: high-effort
4 subset: ko_hle
Source: Horangi Korean Leaderboard 4 data docs
gemini-3.1-pro-preview: high-effort
4 subset: ko_moral
Source: Horangi Korean Leaderboard 4 data docs
gemini-3.1-pro-preview: high-effort
4 subset: ko_truthful_qa
Source: Horangi Korean Leaderboard 4 data docs
gemini-3.1-pro-preview: high-effort
4 subset: kobbq
Source: Horangi Korean Leaderboard 4 data docs
gemini-3.1-pro-preview: high-effort
4 subset: korean_hate_speech
Source: Horangi Korean Leaderboard 4 data docs
gemini-3.1-pro-preview: high-effort
4 subset: squad_kor_v1
Source: Horangi Korean Leaderboard 4 data docs
gemini-3.1-pro-preview: high-effort
4: ALT_유해성방지
Source: Horangi Korean Leaderboard 4 data docs
gemini-3.1-pro-preview: high-effort
4: ALT_윤리/도덕
Source: Horangi Korean Leaderboard 4 data docs
gemini-3.1-pro-preview: high-effort
4: ALT_제어성
Source: Horangi Korean Leaderboard 4 data docs
gemini-3.1-pro-preview: high-effort
4: ALT_편향성방지
Source: Horangi Korean Leaderboard 4 data docs
gemini-3.1-pro-preview: high-effort
4: FINAL_SCORE
Source: Horangi Korean Leaderboard 4 data docs
gemini-3.1-pro-preview: high-effort
4: GLP_구문해석
Source: Horangi Korean Leaderboard 4 data docs
gemini-3.1-pro-preview: high-effort
4: GLP_논리적추론
Source: Horangi Korean Leaderboard 4 data docs
gemini-3.1-pro-preview: high-effort
4: GLP_수학적추론
Source: Horangi Korean Leaderboard 4 data docs
gemini-3.1-pro-preview: high-effort
4: GLP_의미해석
Source: Horangi Korean Leaderboard 4 data docs
gemini-3.1-pro-preview: high-effort
4: GLP_일반적지식
Source: Horangi Korean Leaderboard 4 data docs
gemini-3.1-pro-preview: high-effort
4: GLP_전문적지식
Source: Horangi Korean Leaderboard 4 data docs
gemini-3.1-pro-preview: high-effort
4: GLP_정보검색
Source: Horangi Korean Leaderboard 4 data docs
gemini-3.1-pro-preview: high-effort
4: GLP_추상적추론
Source: Horangi Korean Leaderboard 4 data docs
gemini-3.1-pro-preview: high-effort
4: GLP_코딩능력
Source: Horangi Korean Leaderboard 4 data docs
gemini-3.1-pro-preview: high-effort
4: GLP_표현
Source: Horangi Korean Leaderboard 4 data docs
gemini-3.1-pro-preview: high-effort
4: GLP_함수호출
Source: Horangi Korean Leaderboard 4 data docs
gemini-3.1-pro-preview: high-effort
4: 가치정렬성능(ALT)_AVG
Source: Horangi Korean Leaderboard 4 data docs
gemini-3.1-pro-preview: high-effort
4: 범용언어성능(GLP)_AVG
Source: Horangi Korean Leaderboard 4 data docs
gemini-3.1-pro-preview: high-effort
MentalHealthBench17 related benchmarks · 17 results
Overall
Reported by OpenAI · via BenchBoard docs
OpenAI evaluation · 95% CI available on source page
Acuity · Emergent
Reported by OpenAI · via BenchBoard docs
OpenAI evaluation · 95% CI available on source page
Acuity · High acuity
Reported by OpenAI · via BenchBoard docs
OpenAI evaluation · 95% CI available on source page
Acuity · Non-acute
Reported by OpenAI · via BenchBoard docs
OpenAI evaluation · 95% CI available on source page
Behavior · Actionable guidance and coping
Reported by OpenAI · via BenchBoard docs
OpenAI evaluation · 95% CI available on source page
Behavior · Alarmism and under-response
Reported by OpenAI · via BenchBoard docs
OpenAI evaluation · 95% CI available on source page
Show 11 more resultsHide 11 results
Behavior · Collaboration and user agency
Reported by OpenAI · via BenchBoard docs
OpenAI evaluation · 95% CI available on source page
Behavior · Communication and writing
Reported by OpenAI · via BenchBoard docs
OpenAI evaluation · 95% CI available on source page
Behavior · Context seeking and assessment
Reported by OpenAI · via BenchBoard docs
OpenAI evaluation · 95% CI available on source page
Behavior · Empathy and emotional support
Reported by OpenAI · via BenchBoard docs
OpenAI evaluation · 95% CI available on source page
Behavior · Interpretation and cognitive reframing
Reported by OpenAI · via BenchBoard docs
OpenAI evaluation · 95% CI available on source page
Behavior · Reality-testing calibration
Reported by OpenAI · via BenchBoard docs
OpenAI evaluation · 95% CI available on source page
Behavior · Refusals and harm avoidance
Reported by OpenAI · via BenchBoard docs
OpenAI evaluation · 95% CI available on source page
User profile · Caregiver
Reported by OpenAI · via BenchBoard docs
OpenAI evaluation · 95% CI available on source page
User profile · Clinician
Reported by OpenAI · via BenchBoard docs
OpenAI evaluation · 95% CI available on source page
User profile · Layperson
Reported by OpenAI · via BenchBoard docs
OpenAI evaluation · 95% CI available on source page
User profile · Teen
Reported by OpenAI · via BenchBoard docs
OpenAI evaluation · 95% CI available on source page
MM-ClawBenchMiniMax M2.7 official benchmark overview · exact harness/version not stated · Reported by MiniMax · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Reported by Google DeepMind · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
Gemini 3.5 Flash Model Card · May 2026 · 1M pointwise
Gemini 3.6 Flash Model Card · July 2026 · 1M pointwise
Reported by Meta · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
high · 1M window · 512K–1M bin · 8-needle · mean sequence-matcher ratio
Nejumi 428 related benchmarks · 28 results
NovaPPTBenchSenseNova 6.7 Flash-Lite official benchmark chart · internal benchmark · provider states it will be open-sourced · Reported by SenseTime / OpenSenseNova · via BenchBoard docs · Reported by the model's developer. The source relaying it did not verify it.
OmniCapBench20 related benchmarks · 20 results
Structural · Shot F1
Reported by OmniCapBench authors / Tencent Hunyuan · via BenchBoard docs
OmniCapBench benchmark-owner run · 786 audio-video clips (12.8 hours) · structured MTSS captions · macro average across videos · rule-based structural track with LLM-assisted reference matching · candidate reasoning effort, exact API snapshot and judge model not specified on public leaderboard
Semantic · Non-dialogue event
Reported by OmniCapBench authors / Tencent Hunyuan · via BenchBoard docs
OmniCapBench benchmark-owner run · 786 audio-video clips (12.8 hours) · structured MTSS captions · macro average across videos · bounded localized semantic judging on structurally aligned units · candidate reasoning effort, exact API snapshot and judge model not specified on public leaderboard
Semantic · Scene precision
Reported by OmniCapBench authors / Tencent Hunyuan · via BenchBoard docs
OmniCapBench benchmark-owner run · 786 audio-video clips (12.8 hours) · structured MTSS captions · macro average across videos · bounded localized semantic judging on structurally aligned units · candidate reasoning effort, exact API snapshot and judge model not specified on public leaderboard
Semantic · Scene recall
Reported by OmniCapBench authors / Tencent Hunyuan · via BenchBoard docs
OmniCapBench benchmark-owner run · 786 audio-video clips (12.8 hours) · structured MTSS captions · macro average across videos · bounded localized semantic judging on structurally aligned units · candidate reasoning effort, exact API snapshot and judge model not specified on public leaderboard
Semantic · Shot description
Reported by OmniCapBench authors / Tencent Hunyuan · via BenchBoard docs
OmniCapBench benchmark-owner run · 786 audio-video clips (12.8 hours) · structured MTSS captions · macro average across videos · bounded localized semantic judging on structurally aligned units · candidate reasoning effort, exact API snapshot and judge model not specified on public leaderboard
Semantic · Subject precision
Reported by OmniCapBench authors / Tencent Hunyuan · via BenchBoard docs
OmniCapBench benchmark-owner run · 786 audio-video clips (12.8 hours) · structured MTSS captions · macro average across videos · bounded localized semantic judging on structurally aligned units · candidate reasoning effort, exact API snapshot and judge model not specified on public leaderboard
Show 14 more resultsHide 14 results
Semantic · Subject recall
Reported by OmniCapBench authors / Tencent Hunyuan · via BenchBoard docs
OmniCapBench benchmark-owner run · 786 audio-video clips (12.8 hours) · structured MTSS captions · macro average across videos · bounded localized semantic judging on structurally aligned units · candidate reasoning effort, exact API snapshot and judge model not specified on public leaderboard
Semantic · Subshot precision
Reported by OmniCapBench authors / Tencent Hunyuan · via BenchBoard docs
OmniCapBench benchmark-owner run · 786 audio-video clips (12.8 hours) · structured MTSS captions · macro average across videos · bounded localized semantic judging on structurally aligned units · candidate reasoning effort, exact API snapshot and judge model not specified on public leaderboard
Semantic · Subshot recall
Reported by OmniCapBench authors / Tencent Hunyuan · via BenchBoard docs
OmniCapBench benchmark-owner run · 786 audio-video clips (12.8 hours) · structured MTSS captions · macro average across videos · bounded localized semantic judging on structurally aligned units · candidate reasoning effort, exact API snapshot and judge model not specified on public leaderboard
Semantic table · Dialogue line score
Reported by OmniCapBench authors / Tencent Hunyuan · via BenchBoard docs
OmniCapBench benchmark-owner run · 786 audio-video clips (12.8 hours) · structured MTSS captions · macro average across videos · rule-based dialogue-line measure in the semantic table · candidate reasoning effort, exact API snapshot and judge model not specified on public leaderboard
Structural · Cross-shot coreference
Reported by OmniCapBench authors / Tencent Hunyuan · via BenchBoard docs
OmniCapBench benchmark-owner run · 786 audio-video clips (12.8 hours) · structured MTSS captions · macro average across videos · rule-based structural track with LLM-assisted reference matching · candidate reasoning effort, exact API snapshot and judge model not specified on public leaderboard
Structural · Event-shot association F1
Reported by OmniCapBench authors / Tencent Hunyuan · via BenchBoard docs
OmniCapBench benchmark-owner run · 786 audio-video clips (12.8 hours) · structured MTSS captions · macro average across videos · rule-based structural track with LLM-assisted reference matching · candidate reasoning effort, exact API snapshot and judge model not specified on public leaderboard
Structural · Reference ID usage
Reported by OmniCapBench authors / Tencent Hunyuan · via BenchBoard docs
OmniCapBench benchmark-owner run · 786 audio-video clips (12.8 hours) · structured MTSS captions · macro average across videos · rule-based structural track with LLM-assisted reference matching · candidate reasoning effort, exact API snapshot and judge model not specified on public leaderboard
Structural · Scene reference F1
Reported by OmniCapBench authors / Tencent Hunyuan · via BenchBoard docs
OmniCapBench benchmark-owner run · 786 audio-video clips (12.8 hours) · structured MTSS captions · macro average across videos · rule-based structural track with LLM-assisted reference matching · candidate reasoning effort, exact API snapshot and judge model not specified on public leaderboard
Structural · Schema Gate Compliance
Reported by OmniCapBench authors / Tencent Hunyuan · via BenchBoard docs
OmniCapBench benchmark-owner run · 786 audio-video clips (12.8 hours) · structured MTSS captions · macro average across videos · rule-based structural track with LLM-assisted reference matching · candidate reasoning effort, exact API snapshot and judge model not specified on public leaderboard
Structural · Shot tIoU
Reported by OmniCapBench authors / Tencent Hunyuan · via BenchBoard docs
OmniCapBench benchmark-owner run · 786 audio-video clips (12.8 hours) · structured MTSS captions · macro average across videos · rule-based structural track with LLM-assisted reference matching · candidate reasoning effort, exact API snapshot and judge model not specified on public leaderboard
Structural · Speaker F1
Reported by OmniCapBench authors / Tencent Hunyuan · via BenchBoard docs
OmniCapBench benchmark-owner run · 786 audio-video clips (12.8 hours) · structured MTSS captions · macro average across videos · rule-based structural track with LLM-assisted reference matching · candidate reasoning effort, exact API snapshot and judge model not specified on public leaderboard
Structural · Subject reference F1
Reported by OmniCapBench authors / Tencent Hunyuan · via BenchBoard docs
OmniCapBench benchmark-owner run · 786 audio-video clips (12.8 hours) · structured MTSS captions · macro average across videos · rule-based structural track with LLM-assisted reference matching · candidate reasoning effort, exact API snapshot and judge model not specified on public leaderboard
Structural · Subshot F1
Reported by OmniCapBench authors / Tencent Hunyuan · via BenchBoard docs
OmniCapBench benchmark-owner run · 786 audio-video clips (12.8 hours) · structured MTSS captions · macro average across videos · rule-based structural track with LLM-assisted reference matching · candidate reasoning effort, exact API snapshot and judge model not specified on public leaderboard
Structural · Subshot tIoU
Reported by OmniCapBench authors / Tencent Hunyuan · via BenchBoard docs
OmniCapBench benchmark-owner run · 786 audio-video clips (12.8 hours) · structured MTSS captions · macro average across videos · rule-based structural track with LLM-assisted reference matching · candidate reasoning effort, exact API snapshot and judge model not specified on public leaderboard
PerceptionBench9 related benchmarks · 9 results
Overall accuracy
Reported by Moonshot AI / PerceptionBench authors · via BenchBoard docs
benchmark-owner evaluation · 3,000 verified questions total · open-ended short answers · unified prompts · high reasoning · main-table result; run count not specified · GPT-oss-120B judge
Attribute accuracy
Reported by Moonshot AI / PerceptionBench authors · via BenchBoard docs
benchmark-owner evaluation · 3,000 verified questions total · open-ended short answers · unified prompts · high reasoning · main-table result; run count not specified · GPT-oss-120B judge
Comparison accuracy
Reported by Moonshot AI / PerceptionBench authors · via BenchBoard docs
benchmark-owner evaluation · 3,000 verified questions total · open-ended short answers · unified prompts · high reasoning · main-table result; run count not specified · GPT-oss-120B judge
Contextual integration accuracy
Reported by Moonshot AI / PerceptionBench authors · via BenchBoard docs
benchmark-owner evaluation · 3,000 verified questions total · open-ended short answers · unified prompts · high reasoning · main-table result; run count not specified · GPT-oss-120B judge
Counting accuracy
Reported by Moonshot AI / PerceptionBench authors · via BenchBoard docs
benchmark-owner evaluation · 3,000 verified questions total · open-ended short answers · unified prompts · high reasoning · main-table result; run count not specified · GPT-oss-120B judge
Depth and 3D perception accuracy
Reported by Moonshot AI / PerceptionBench authors · via BenchBoard docs
benchmark-owner evaluation · 3,000 verified questions total · open-ended short answers · unified prompts · high reasoning · main-table result; run count not specified · GPT-oss-120B judge
Show 3 more resultsHide 3 results
Fine-grained recognition accuracy
Reported by Moonshot AI / PerceptionBench authors · via BenchBoard docs
benchmark-owner evaluation · 3,000 verified questions total · open-ended short answers · unified prompts · high reasoning · main-table result; run count not specified · GPT-oss-120B judge
Localization accuracy
Reported by Moonshot AI / PerceptionBench authors · via BenchBoard docs
benchmark-owner evaluation · 3,000 verified questions total · open-ended short answers · unified prompts · high reasoning · main-table result; run count not specified · GPT-oss-120B judge
Visual relation accuracy
Reported by Moonshot AI / PerceptionBench authors · via BenchBoard docs
benchmark-owner evaluation · 3,000 verified questions total · open-ended short answers · unified prompts · high reasoning · main-table result; run count not specified · GPT-oss-120B judge
RNG-Bench12 related benchmarks · 12 results
Duel Elo
Reported by RNG-Bench / InternLM · via BenchBoard docs
Matching Pairs · image · poker theme · 16 games per model
3D Maze Efficiency
Reported by RNG-Bench / InternLM · via BenchBoard docs
successful episodes only · 13×13 · no minimap · mean optimal path 60 steps
3D Maze Explore Rate
Reported by RNG-Bench / InternLM · via BenchBoard docs
single-player · 13×13 · no minimap · mean optimal path 60 steps
3D Maze Game Score
Reported by RNG-Bench / InternLM · via BenchBoard docs
single-player · 13×13 · no minimap · mean optimal path 60 steps
3D Maze Success Rate
Reported by RNG-Bench / InternLM · via BenchBoard docs
single-player · 13×13 · no minimap · mean optimal path 60 steps
3D Maze Wall Collisions
Reported by RNG-Bench / InternLM · via BenchBoard docs
single-player · 13×13 · no minimap · mean optimal path 60 steps
Show 6 more resultsHide 6 results
Duel Score
Reported by RNG-Bench / InternLM · via BenchBoard docs
Matching Pairs · image · poker theme · 16 games per model
Duel Win Rate
Reported by RNG-Bench / InternLM · via BenchBoard docs
Matching Pairs · image · poker theme · 16 games per model
Matching Pairs Invalid Action
Reported by RNG-Bench / InternLM · via BenchBoard docs
single-player · 10×10 · image · noise theme
Matching Pairs Parse Failure
Reported by RNG-Bench / InternLM · via BenchBoard docs
single-player · 10×10 · image · noise theme
Matching Pairs Responses per Score
Reported by RNG-Bench / InternLM · via BenchBoard docs
single-player · 10×10 · image · noise theme
Matching Pairs Score
Reported by RNG-Bench / InternLM · via BenchBoard docs
single-player · 10×10 · image · noise theme
Vals16 related benchmarks · 18 results
Show 12 more resultsHide 12 results
MedCode
Source: Clinical Benchmarks data docs · Sep 2026
Setting not stated
Source: Vals AI
Setting not stated
MedScribe
Source: Clinical Benchmarks data docs · Sep 2026
Setting not stated
Source: Vals AI
Setting not stated
MortgageTax
Source: Vals AI

Data credits: BenchBoard by SYuan03 (github.com/SYuan03/benchboard). MIT covers its code and data organization; no results licence is inferred. Numeric relays retain the original stating-source links. · Epoch AI Benchmarking Hub, CC BY 4.0 (licence) · Data by LLM Stats · LMArena, CC BY 4.0 (licence) · ArmBench-LLM by Metric-AI. Public numeric primary task results; no separate results licence asserted. · Bangla LLM Bench, Md Deluair Hossen. Source tasks: Hendrycks et al. 2021 (MMLU), OpenAI MMMLU, Bandarkar et al. 2024 (Belebele), Shi et al. 2023 (MGSM), Nahin et al. 2025 (TituLLMs). Public numeric community results; results licence not separately stated. · BullshitBench by Peter Gostev, MIT (licence) · EQ-Bench (Samuel J. Paech) — cite paech2023eqbench · CyberGym leaderboard (Wang, Shi, He, Cai, Zhang, Song; UC Berkeley), ICLR 2026. Results evaluated and submitted by the listed teams. No licence is stated for the result data; the repository's Apache-2.0 licence covers its code only. · Keelemudelite mõõdupuu: Independent LLM leaderboard for Estonian. Eesti Keele Instituut. https://moodupuu.eki.ee, CC BY 4.0. Latest per-benchmark results selected with partial-run flags retained. (licence) · ExploitGym leaderboard (Wang et al., UC Berkeley), arXiv:2605.11086. No licence is stated for the result data; the repository's Apache-2.0 licence covers its code only, and task data keeps its upstream licences. Literal exploit counts; hidden rows omitted. · Clinical Benchmarks (clinicalbenchmarks.ai), CC BY 4.0; fractions converted to percentages; source documents retain their own terms (licence) · OpenRouter public benchmark leaderboard and search-lane exports, CC BY 4.0. Fractional source scores converted to percentages; immutable dated snapshots. (licence) · HalluHard: Fan, Delsad, Flammarion and Andriushchenko (EPFL), arXiv:2602.01031 · Steel.dev Agent Leaderboards, MIT (licence) · Horangi 4, Weights & Biases Korea; public publisher-selected numeric evaluation results · JMLE2026-Bench and IgakuQA119 by Naoto Iwase; results licence not stated (code MIT, question data CC BY 4.0). · LLM Chess by Maxim Saplin, Apache-2.0 (Copyright [2024] [Maxim Saplin]) (licence) · MineBench, MIT (licence) · Nejumi Leaderboard 4, Weights & Biases Japan; public publisher-selected numeric evaluation results · Outback Yak Research, NepNewsBench v2 research draft. Public numerical aggregates from a candidate preview that the publisher says is not evidence of publication; no release licence assigned and none asserted here. · OpenHands Index (All Hands AI), Apache-2.0 (licence) · Phare (Giskard), CC BY 4.0 (licence) · SnakeBench (Greg Kamradt / ARC Prize Foundation), MIT (licence) · SpreadsheetBench (Renmin University of China and contributors), spreadsheetbench.github.io. · SWE-bench-Live (Microsoft and contributors), swe-bench-live.github.io. Resolve rates computed from the published report counts. · Sierra Research tau2-bench leaderboard submissions (github.com/sierra-research/tau2-bench), MIT (licence) · UzLiB, Tahrirchi (Abror Shopulatov). Public LEADERBOARD.md numbers; repository MIT covers software and documentation, no separate results licence stated and none asserted. · Vectara Hallucination Leaderboard, Apache-2.0 (licence)

Also known as: aihubmix/gemini-3.1-pro-preview, aihubmix/gemini-3.1-pro-preview-customtools, databricks-gemini-3-1-pro, databricks/databricks-gemini-3-1-pro, deepinfra/google/gemini-3.1-pro, gemini-3-1-pro, gemini-3-1-pro-preview, gemini-3.1-pro, gemini-3.1-pro-grounding, gemini-3.1-pro-preview-customtools, gemini-3.1-pro-preview-search, gemini-3.1-pro-preview-thinking, gemini-3.1-pro-preview:web, gemini/gemini-3.1-pro-preview, gemini/gemini-3.1-pro-preview-customtools, gmi/google/gemini-3.1-pro-preview, google-ai-studio/gemini-3.1-pro-preview, google-gemini-3.1-pro-preview, google-vertex/gemini-3.1-pro-preview, google/gemini-3.1-pro, google/gemini-3.1-pro-preview, google/gemini-3.1-pro-preview-batch, google/gemini-3.1-pro-preview-customtools, google/gemini-3.1-pro-preview-high, google/gemini-3.1-pro-preview-low, google/gemini-3.1-pro-preview:batch, modelrunner-ai/google-gemini-3.1-pro, perplexity/google/gemini-3.1-pro-preview, replicate/google/gemini-3.1-pro, vertex/gemini-3.1-pro-preview, vertex/gemini-3.1-pro-preview:flex, vertex_ai/gemini-3.1-pro-preview, vertex_ai/gemini-3.1-pro-preview-customtools

Like this one. Tap or hover any dotted value, label or icon to see where it comes from and how it is measured. open details when tapped or hovered. How this data works