Skip to content

OpenAI · Chat

GPT 4o

OpenAI 🇺🇸ChatGPT familyOne of its sources says the weights aren't publicly available.The maker currently plans to deprecate it within the next 180 days (14 Jan 2027). Plans can change.
From Netmind's listing:
GPT-4o is a fast, multimodal flagship model from OpenAI that accepts text and image inputs and returns text. It offers strong general reasoning, reliable function calling, and a 128K-token context window, well suited for chat, coding, analysis, and everyday production workloads.
llminfo IQ
113.7
Max context
131K
Listings
134

Evidence and sources

llminfo IQ 113.7, ranked among chat models. From 761 benchmarks. Plenty of data behind it.How llminfo IQ is computedCoding IQ 77.4, 95% interval 67–87. From 13 benchmark families across 14 sources. Plenty of data behind it. Uses the model's best published results. It has its own scale, separate from llminfo IQ.How Coding IQ is computedPoor value (24.6). Costs more per task than its benchmark results predict. From 23 benchmarks with measured task costs. Plenty of data behind it.How Value for money worksReleased 13 May 2024 Training-data cutoff, as its sources state it Maker HQ: US (according to OpenAI's own website)
All rankings: #153 on NanoGPT recently #215 on OpenRouter today #217 on OpenRouter this week #224 on OpenRouter this monthSources: OpenAI API data, Inference.net data, LMArena data, LiteLLM data, Models.dev data, AI/ML API data, OpenRouter data docs, NetMind.AI, llminfo researchPrimary data from OpenAI API from Inference.net: name from LMArena: variant from LiteLLM: deprecation date from Models.dev: weights availability from AI/ML API: primary mode from OpenRouter: knowledge cutoff from NetMind.AI: description from llminfo research: release date
Tags from its sources:instructtoolsvisionreasoningdocumentmultimodalfunction-callinggeneral-purpose

Advantages and trade-offs

Compared with 10 chat models, picked mainly by popularity and IQ

Trade-offs

Deprecation scheduled

The maker plans to deprecate it on 14 Jan 2027, within the next 180 days.

We only make a claim when there's enough data on models of the same type. Updated 22 Nov, 06:07 UTC.Prices come from the model's maker where they sell it, otherwise their official partner, otherwise the market median. This comparison uses 9 maker's prices; 1 market median price.

Input & output

as stated by the model's providers
Text
Images
Audio
Files
Text
Audio

Capabilities

Supported operations and provider-stated features

VisionReasoningFunction callingStructured outputPrompt cachingPDF inputWeb searchAudio inputBatch API

Listings

 OpenAIgpt-4oSources: GenAI Prices data, LiteLLM data, Models.dev data, OpenAI API data128K$2.50cache read $1.25batch $1.25$10.00batch $5.00
 Microsoft Azuregpt-4o-batch · batchSource: Helicone Costs dataNot published by the provider$1.25$5.00
 OpenRouteropenai/gpt-4o:batch · batchSource: OpenRouter data docs128K$1.25cache read $0.625$5.00
 Vercelopenai/gpt-4o-batch · batchSource: Vercel AI Gateway data128K$1.25cache read $0.625$5.00
 Cloudflare AI Gatewayopenai/gpt-4oSource: Models.dev data128K$1.25cache read $0.625$5.00
 Poeopenai/gpt-4o-search · searchSource: Models.dev data128K$2.2727$9.0909
 302aigpt-4oSources: 302.ai data, Models.dev data128K$2.50$10.00
 Aihubmixgpt-4oSource: AIHubMix data docs128K$2.50cache read $1.25$10.00
 Cometapigpt-4oSources: CometAPI data, CometAPI Pricing dataNot published by the provider$2.50$10.00
 Eden AIInference by OpenAIopenai/gpt-4oSource: Models.dev data128K$2.50cache read $1.25$10.00

24 listings without a price are hidden.

Benchmarks

Named effort levels run from lowest to highest. Select a result to see its source details.

Multilingual

Bridging the Gap533 related benchmarks · 549 results
Belebele English (en) original
Source: Bridging the Gap (arXiv:2412.12417v1) data
By section · table: A.15 · task: Belebele · language: en · shots: 0
table: 1, section: baseline, task: Belebele
table: 2, task: Belebele, language: en, shots: 0
Belebele 11-language African average
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 1, section: average, task: Belebele
Belebele Afrikaans (af) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-accuracy, task: Belebele, language: af, shots: 0
Belebele Afrikaans (af) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: Belebele, language: af, shots: 0
Belebele Afrikaans (af) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-difference, task: Belebele, language: af, shots: 0
Belebele Afrikaans (af) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-difference, task: Belebele, language: af, shots: 0
Show 540 more resultsHide 540 results
Belebele Afrikaans (af) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-accuracy, task: Belebele, language: af, shots: 0
Belebele Amharic (am) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-accuracy, task: Belebele, language: am, shots: 0
Belebele Amharic (am) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: Belebele, language: am, shots: 0
Belebele Amharic (am) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-difference, task: Belebele, language: am, shots: 0
Belebele Amharic (am) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-difference, task: Belebele, language: am, shots: 0
Belebele Amharic (am) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-accuracy, task: Belebele, language: am, shots: 0
Belebele Bambara (bm) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-accuracy, task: Belebele, language: bm, shots: 0
Belebele Bambara (bm) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: Belebele, language: bm, shots: 0
Belebele Bambara (bm) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-difference, task: Belebele, language: bm, shots: 0
Belebele Bambara (bm) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-difference, task: Belebele, language: bm, shots: 0
Belebele Bambara (bm) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-accuracy, task: Belebele, language: bm, shots: 0
Belebele English (en) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-difference, task: Belebele, language: en, shots: 0
Belebele English (en) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-difference, task: Belebele, language: en, shots: 0
Belebele Igbo (ig) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-accuracy, task: Belebele, language: ig, shots: 0
Belebele Igbo (ig) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: Belebele, language: ig, shots: 0
Belebele Igbo (ig) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-difference, task: Belebele, language: ig, shots: 0
Belebele Igbo (ig) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-difference, task: Belebele, language: ig, shots: 0
Belebele Igbo (ig) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-accuracy, task: Belebele, language: ig, shots: 0
Belebele Sepedi (nso) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-accuracy, task: Belebele, language: nso, shots: 0
Belebele Sepedi (nso) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: Belebele, language: nso, shots: 0
Belebele Sepedi (nso) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-difference, task: Belebele, language: nso, shots: 0
Belebele Sepedi (nso) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-difference, task: Belebele, language: nso, shots: 0
Belebele Sepedi (nso) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-accuracy, task: Belebele, language: nso, shots: 0
Belebele Sesotho (st) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-accuracy, task: Belebele, language: st, shots: 0
Belebele Sesotho (st) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: Belebele, language: st, shots: 0
Belebele Sesotho (st) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-difference, task: Belebele, language: st, shots: 0
Belebele Sesotho (st) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-difference, task: Belebele, language: st, shots: 0
Belebele Sesotho (st) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-accuracy, task: Belebele, language: st, shots: 0
Belebele Setswana (tn) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-accuracy, task: Belebele, language: tn, shots: 0
Belebele Setswana (tn) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: Belebele, language: tn, shots: 0
Belebele Setswana (tn) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-difference, task: Belebele, language: tn, shots: 0
Belebele Setswana (tn) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-difference, task: Belebele, language: tn, shots: 0
Belebele Setswana (tn) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-accuracy, task: Belebele, language: tn, shots: 0
Belebele Shona (sn) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-accuracy, task: Belebele, language: sn, shots: 0
Belebele Shona (sn) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: Belebele, language: sn, shots: 0
Belebele Shona (sn) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-difference, task: Belebele, language: sn, shots: 0
Belebele Shona (sn) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-difference, task: Belebele, language: sn, shots: 0
Belebele Shona (sn) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-accuracy, task: Belebele, language: sn, shots: 0
Belebele Tsonga (ts) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-accuracy, task: Belebele, language: ts, shots: 0
Belebele Tsonga (ts) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: Belebele, language: ts, shots: 0
Belebele Tsonga (ts) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-difference, task: Belebele, language: ts, shots: 0
Belebele Tsonga (ts) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-difference, task: Belebele, language: ts, shots: 0
Belebele Tsonga (ts) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-accuracy, task: Belebele, language: ts, shots: 0
Belebele Xhosa (xh) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-accuracy, task: Belebele, language: xh, shots: 0
Belebele Xhosa (xh) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: Belebele, language: xh, shots: 0
Belebele Xhosa (xh) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-difference, task: Belebele, language: xh, shots: 0
Belebele Xhosa (xh) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-difference, task: Belebele, language: xh, shots: 0
Belebele Xhosa (xh) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-accuracy, task: Belebele, language: xh, shots: 0
Belebele Zulu (zu) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-accuracy, task: Belebele, language: zu, shots: 0
Belebele Zulu (zu) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: Belebele, language: zu, shots: 0
Belebele Zulu (zu) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: back-translation-difference, task: Belebele, language: zu, shots: 0
Belebele Zulu (zu) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-difference, task: Belebele, language: zu, shots: 0
Belebele Zulu (zu) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.15, section: machine-translation-accuracy, task: Belebele, language: zu, shots: 0
Clinical Knowledge Afrikaans (af) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-difference, task: MMLU Clinical Knowledge, language: af, shots: 5
Clinical Knowledge Afrikaans (af) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-difference, task: MMLU Clinical Knowledge, language: af, shots: 5
Clinical Knowledge Amharic (am) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-difference, task: MMLU Clinical Knowledge, language: am, shots: 5
Clinical Knowledge Amharic (am) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-difference, task: MMLU Clinical Knowledge, language: am, shots: 5
Clinical Knowledge Bambara (bm) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-difference, task: MMLU Clinical Knowledge, language: bm, shots: 5
Clinical Knowledge Bambara (bm) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-difference, task: MMLU Clinical Knowledge, language: bm, shots: 5
Clinical Knowledge English (en) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-difference, task: MMLU Clinical Knowledge, language: en, shots: 5
Clinical Knowledge English (en) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-difference, task: MMLU Clinical Knowledge, language: en, shots: 5
Clinical Knowledge Igbo (ig) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-difference, task: MMLU Clinical Knowledge, language: ig, shots: 5
Clinical Knowledge Igbo (ig) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-difference, task: MMLU Clinical Knowledge, language: ig, shots: 5
Clinical Knowledge Sepedi (nso) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-difference, task: MMLU Clinical Knowledge, language: nso, shots: 5
Clinical Knowledge Sepedi (nso) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-difference, task: MMLU Clinical Knowledge, language: nso, shots: 5
Clinical Knowledge Sesotho (st) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-difference, task: MMLU Clinical Knowledge, language: st, shots: 5
Clinical Knowledge Sesotho (st) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-difference, task: MMLU Clinical Knowledge, language: st, shots: 5
Clinical Knowledge Setswana (tn) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-difference, task: MMLU Clinical Knowledge, language: tn, shots: 5
Clinical Knowledge Setswana (tn) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-difference, task: MMLU Clinical Knowledge, language: tn, shots: 5
Clinical Knowledge Shona (sn) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-difference, task: MMLU Clinical Knowledge, language: sn, shots: 5
Clinical Knowledge Shona (sn) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-difference, task: MMLU Clinical Knowledge, language: sn, shots: 5
Clinical Knowledge Tsonga (ts) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-difference, task: MMLU Clinical Knowledge, language: ts, shots: 5
Clinical Knowledge Tsonga (ts) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-difference, task: MMLU Clinical Knowledge, language: ts, shots: 5
Clinical Knowledge Xhosa (xh) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-difference, task: MMLU Clinical Knowledge, language: xh, shots: 5
Clinical Knowledge Xhosa (xh) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-difference, task: MMLU Clinical Knowledge, language: xh, shots: 5
Clinical Knowledge Zulu (zu) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-difference, task: MMLU Clinical Knowledge, language: zu, shots: 5
Clinical Knowledge Zulu (zu) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-difference, task: MMLU Clinical Knowledge, language: zu, shots: 5
College Medicine Afrikaans (af) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-difference, task: MMLU College Medicine, language: af, shots: 5
College Medicine Afrikaans (af) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-difference, task: MMLU College Medicine, language: af, shots: 5
College Medicine Amharic (am) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-difference, task: MMLU College Medicine, language: am, shots: 5
College Medicine Amharic (am) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-difference, task: MMLU College Medicine, language: am, shots: 5
College Medicine Bambara (bm) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-difference, task: MMLU College Medicine, language: bm, shots: 5
College Medicine Bambara (bm) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-difference, task: MMLU College Medicine, language: bm, shots: 5
College Medicine English (en) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-difference, task: MMLU College Medicine, language: en, shots: 5
College Medicine English (en) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-difference, task: MMLU College Medicine, language: en, shots: 5
College Medicine Igbo (ig) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-difference, task: MMLU College Medicine, language: ig, shots: 5
College Medicine Igbo (ig) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-difference, task: MMLU College Medicine, language: ig, shots: 5
College Medicine Sepedi (nso) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-difference, task: MMLU College Medicine, language: nso, shots: 5
College Medicine Sepedi (nso) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-difference, task: MMLU College Medicine, language: nso, shots: 5
College Medicine Sesotho (st) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-difference, task: MMLU College Medicine, language: st, shots: 5
College Medicine Sesotho (st) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-difference, task: MMLU College Medicine, language: st, shots: 5
College Medicine Setswana (tn) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-difference, task: MMLU College Medicine, language: tn, shots: 5
College Medicine Setswana (tn) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-difference, task: MMLU College Medicine, language: tn, shots: 5
College Medicine Shona (sn) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-difference, task: MMLU College Medicine, language: sn, shots: 5
College Medicine Shona (sn) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-difference, task: MMLU College Medicine, language: sn, shots: 5
College Medicine Tsonga (ts) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-difference, task: MMLU College Medicine, language: ts, shots: 5
College Medicine Tsonga (ts) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-difference, task: MMLU College Medicine, language: ts, shots: 5
College Medicine Xhosa (xh) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-difference, task: MMLU College Medicine, language: xh, shots: 5
College Medicine Xhosa (xh) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-difference, task: MMLU College Medicine, language: xh, shots: 5
College Medicine Zulu (zu) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-difference, task: MMLU College Medicine, language: zu, shots: 5
College Medicine Zulu (zu) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-difference, task: MMLU College Medicine, language: zu, shots: 5
MMLU Clinical Knowledge 11-language African average
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 1, section: average, task: MMLU Clinical Knowledge
MMLU Clinical Knowledge Afrikaans (af) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-accuracy, task: MMLU Clinical Knowledge, language: af, shots: 5
MMLU Clinical Knowledge Afrikaans (af) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU Clinical Knowledge, language: af, shots: 5
MMLU Clinical Knowledge Afrikaans (af) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-accuracy, task: MMLU Clinical Knowledge, language: af, shots: 5
MMLU Clinical Knowledge Amharic (am) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-accuracy, task: MMLU Clinical Knowledge, language: am, shots: 5
MMLU Clinical Knowledge Amharic (am) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU Clinical Knowledge, language: am, shots: 5
MMLU Clinical Knowledge Amharic (am) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-accuracy, task: MMLU Clinical Knowledge, language: am, shots: 5
MMLU Clinical Knowledge Bambara (bm) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-accuracy, task: MMLU Clinical Knowledge, language: bm, shots: 5
MMLU Clinical Knowledge Bambara (bm) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU Clinical Knowledge, language: bm, shots: 5
MMLU Clinical Knowledge Bambara (bm) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-accuracy, task: MMLU Clinical Knowledge, language: bm, shots: 5
MMLU Clinical Knowledge English (en) original
Source: Bridging the Gap (arXiv:2412.12417v1) data
By section · table: A.13 · task: MMLU Clinical Knowledge · language: en · shots: 5
table: 1, section: baseline, task: MMLU Clinical Knowledge
table: 2, task: MMLU Clinical Knowledge, language: en, shots: 5
MMLU Clinical Knowledge Igbo (ig) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-accuracy, task: MMLU Clinical Knowledge, language: ig, shots: 5
MMLU Clinical Knowledge Igbo (ig) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU Clinical Knowledge, language: ig, shots: 5
MMLU Clinical Knowledge Igbo (ig) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-accuracy, task: MMLU Clinical Knowledge, language: ig, shots: 5
MMLU Clinical Knowledge Sepedi (nso) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-accuracy, task: MMLU Clinical Knowledge, language: nso, shots: 5
MMLU Clinical Knowledge Sepedi (nso) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU Clinical Knowledge, language: nso, shots: 5
MMLU Clinical Knowledge Sepedi (nso) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-accuracy, task: MMLU Clinical Knowledge, language: nso, shots: 5
MMLU Clinical Knowledge Sesotho (st) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-accuracy, task: MMLU Clinical Knowledge, language: st, shots: 5
MMLU Clinical Knowledge Sesotho (st) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU Clinical Knowledge, language: st, shots: 5
MMLU Clinical Knowledge Sesotho (st) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-accuracy, task: MMLU Clinical Knowledge, language: st, shots: 5
MMLU Clinical Knowledge Setswana (tn) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-accuracy, task: MMLU Clinical Knowledge, language: tn, shots: 5
MMLU Clinical Knowledge Setswana (tn) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU Clinical Knowledge, language: tn, shots: 5
MMLU Clinical Knowledge Setswana (tn) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-accuracy, task: MMLU Clinical Knowledge, language: tn, shots: 5
MMLU Clinical Knowledge Shona (sn) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-accuracy, task: MMLU Clinical Knowledge, language: sn, shots: 5
MMLU Clinical Knowledge Shona (sn) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU Clinical Knowledge, language: sn, shots: 5
MMLU Clinical Knowledge Shona (sn) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-accuracy, task: MMLU Clinical Knowledge, language: sn, shots: 5
MMLU Clinical Knowledge Tsonga (ts) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-accuracy, task: MMLU Clinical Knowledge, language: ts, shots: 5
MMLU Clinical Knowledge Tsonga (ts) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU Clinical Knowledge, language: ts, shots: 5
MMLU Clinical Knowledge Tsonga (ts) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-accuracy, task: MMLU Clinical Knowledge, language: ts, shots: 5
MMLU Clinical Knowledge Xhosa (xh) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-accuracy, task: MMLU Clinical Knowledge, language: xh, shots: 5
MMLU Clinical Knowledge Xhosa (xh) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU Clinical Knowledge, language: xh, shots: 5
MMLU Clinical Knowledge Xhosa (xh) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-accuracy, task: MMLU Clinical Knowledge, language: xh, shots: 5
MMLU Clinical Knowledge Zulu (zu) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: back-translation-accuracy, task: MMLU Clinical Knowledge, language: zu, shots: 5
MMLU Clinical Knowledge Zulu (zu) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU Clinical Knowledge, language: zu, shots: 5
MMLU Clinical Knowledge Zulu (zu) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.13, section: machine-translation-accuracy, task: MMLU Clinical Knowledge, language: zu, shots: 5
MMLU College Medicine 11-language African average
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 1, section: average, task: MMLU College Medicine
MMLU College Medicine Afrikaans (af) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-accuracy, task: MMLU College Medicine, language: af, shots: 5
MMLU College Medicine Afrikaans (af) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU College Medicine, language: af, shots: 5
MMLU College Medicine Afrikaans (af) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-accuracy, task: MMLU College Medicine, language: af, shots: 5
MMLU College Medicine Amharic (am) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-accuracy, task: MMLU College Medicine, language: am, shots: 5
MMLU College Medicine Amharic (am) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU College Medicine, language: am, shots: 5
MMLU College Medicine Amharic (am) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-accuracy, task: MMLU College Medicine, language: am, shots: 5
MMLU College Medicine Bambara (bm) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-accuracy, task: MMLU College Medicine, language: bm, shots: 5
MMLU College Medicine Bambara (bm) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU College Medicine, language: bm, shots: 5
MMLU College Medicine Bambara (bm) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-accuracy, task: MMLU College Medicine, language: bm, shots: 5
MMLU College Medicine English (en) original
Source: Bridging the Gap (arXiv:2412.12417v1) data
By section · table: A.12 · task: MMLU College Medicine · language: en · shots: 5
table: 1, section: baseline, task: MMLU College Medicine
table: 2, task: MMLU College Medicine, language: en, shots: 5
MMLU College Medicine Igbo (ig) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-accuracy, task: MMLU College Medicine, language: ig, shots: 5
MMLU College Medicine Igbo (ig) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU College Medicine, language: ig, shots: 5
MMLU College Medicine Igbo (ig) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-accuracy, task: MMLU College Medicine, language: ig, shots: 5
MMLU College Medicine Sepedi (nso) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-accuracy, task: MMLU College Medicine, language: nso, shots: 5
MMLU College Medicine Sepedi (nso) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU College Medicine, language: nso, shots: 5
MMLU College Medicine Sepedi (nso) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-accuracy, task: MMLU College Medicine, language: nso, shots: 5
MMLU College Medicine Sesotho (st) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-accuracy, task: MMLU College Medicine, language: st, shots: 5
MMLU College Medicine Sesotho (st) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU College Medicine, language: st, shots: 5
MMLU College Medicine Sesotho (st) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-accuracy, task: MMLU College Medicine, language: st, shots: 5
MMLU College Medicine Setswana (tn) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-accuracy, task: MMLU College Medicine, language: tn, shots: 5
MMLU College Medicine Setswana (tn) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU College Medicine, language: tn, shots: 5
MMLU College Medicine Setswana (tn) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-accuracy, task: MMLU College Medicine, language: tn, shots: 5
MMLU College Medicine Shona (sn) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-accuracy, task: MMLU College Medicine, language: sn, shots: 5
MMLU College Medicine Shona (sn) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU College Medicine, language: sn, shots: 5
MMLU College Medicine Shona (sn) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-accuracy, task: MMLU College Medicine, language: sn, shots: 5
MMLU College Medicine Tsonga (ts) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-accuracy, task: MMLU College Medicine, language: ts, shots: 5
MMLU College Medicine Tsonga (ts) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU College Medicine, language: ts, shots: 5
MMLU College Medicine Tsonga (ts) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-accuracy, task: MMLU College Medicine, language: ts, shots: 5
MMLU College Medicine Xhosa (xh) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-accuracy, task: MMLU College Medicine, language: xh, shots: 5
MMLU College Medicine Xhosa (xh) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU College Medicine, language: xh, shots: 5
MMLU College Medicine Xhosa (xh) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-accuracy, task: MMLU College Medicine, language: xh, shots: 5
MMLU College Medicine Zulu (zu) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: back-translation-accuracy, task: MMLU College Medicine, language: zu, shots: 5
MMLU College Medicine Zulu (zu) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU College Medicine, language: zu, shots: 5
MMLU College Medicine Zulu (zu) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.12, section: machine-translation-accuracy, task: MMLU College Medicine, language: zu, shots: 5
MMLU Virology 11-language African average
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 1, section: average, task: MMLU Virology
MMLU Virology Afrikaans (af) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-accuracy, task: MMLU Virology, language: af, shots: 5
MMLU Virology Afrikaans (af) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU Virology, language: af, shots: 5
MMLU Virology Afrikaans (af) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-accuracy, task: MMLU Virology, language: af, shots: 5
MMLU Virology Amharic (am) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-accuracy, task: MMLU Virology, language: am, shots: 5
MMLU Virology Amharic (am) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU Virology, language: am, shots: 5
MMLU Virology Amharic (am) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-accuracy, task: MMLU Virology, language: am, shots: 5
MMLU Virology Bambara (bm) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-accuracy, task: MMLU Virology, language: bm, shots: 5
MMLU Virology Bambara (bm) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU Virology, language: bm, shots: 5
MMLU Virology Bambara (bm) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-accuracy, task: MMLU Virology, language: bm, shots: 5
MMLU Virology English (en) original
Source: Bridging the Gap (arXiv:2412.12417v1) data
By section · table: A.14 · task: MMLU Virology · language: en · shots: 5
table: 1, section: baseline, task: MMLU Virology
table: 2, task: MMLU Virology, language: en, shots: 5
MMLU Virology Igbo (ig) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-accuracy, task: MMLU Virology, language: ig, shots: 5
MMLU Virology Igbo (ig) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU Virology, language: ig, shots: 5
MMLU Virology Igbo (ig) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-accuracy, task: MMLU Virology, language: ig, shots: 5
MMLU Virology Sepedi (nso) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-accuracy, task: MMLU Virology, language: nso, shots: 5
MMLU Virology Sepedi (nso) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU Virology, language: nso, shots: 5
MMLU Virology Sepedi (nso) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-accuracy, task: MMLU Virology, language: nso, shots: 5
MMLU Virology Sesotho (st) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-accuracy, task: MMLU Virology, language: st, shots: 5
MMLU Virology Sesotho (st) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU Virology, language: st, shots: 5
MMLU Virology Sesotho (st) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-accuracy, task: MMLU Virology, language: st, shots: 5
MMLU Virology Setswana (tn) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-accuracy, task: MMLU Virology, language: tn, shots: 5
MMLU Virology Setswana (tn) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU Virology, language: tn, shots: 5
MMLU Virology Setswana (tn) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-accuracy, task: MMLU Virology, language: tn, shots: 5
MMLU Virology Shona (sn) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-accuracy, task: MMLU Virology, language: sn, shots: 5
MMLU Virology Shona (sn) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU Virology, language: sn, shots: 5
MMLU Virology Shona (sn) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-accuracy, task: MMLU Virology, language: sn, shots: 5
MMLU Virology Tsonga (ts) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-accuracy, task: MMLU Virology, language: ts, shots: 5
MMLU Virology Tsonga (ts) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU Virology, language: ts, shots: 5
MMLU Virology Tsonga (ts) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-accuracy, task: MMLU Virology, language: ts, shots: 5
MMLU Virology Xhosa (xh) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-accuracy, task: MMLU Virology, language: xh, shots: 5
MMLU Virology Xhosa (xh) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU Virology, language: xh, shots: 5
MMLU Virology Xhosa (xh) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-accuracy, task: MMLU Virology, language: xh, shots: 5
MMLU Virology Zulu (zu) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-accuracy, task: MMLU Virology, language: zu, shots: 5
MMLU Virology Zulu (zu) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: MMLU Virology, language: zu, shots: 5
MMLU Virology Zulu (zu) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-accuracy, task: MMLU Virology, language: zu, shots: 5
Virology Afrikaans (af) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-difference, task: MMLU Virology, language: af, shots: 5
Virology Afrikaans (af) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-difference, task: MMLU Virology, language: af, shots: 5
Virology Amharic (am) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-difference, task: MMLU Virology, language: am, shots: 5
Virology Amharic (am) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-difference, task: MMLU Virology, language: am, shots: 5
Virology Bambara (bm) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-difference, task: MMLU Virology, language: bm, shots: 5
Virology Bambara (bm) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-difference, task: MMLU Virology, language: bm, shots: 5
Virology English (en) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-difference, task: MMLU Virology, language: en, shots: 5
Virology English (en) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-difference, task: MMLU Virology, language: en, shots: 5
Virology Igbo (ig) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-difference, task: MMLU Virology, language: ig, shots: 5
Virology Igbo (ig) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-difference, task: MMLU Virology, language: ig, shots: 5
Virology Sepedi (nso) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-difference, task: MMLU Virology, language: nso, shots: 5
Virology Sepedi (nso) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-difference, task: MMLU Virology, language: nso, shots: 5
Virology Sesotho (st) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-difference, task: MMLU Virology, language: st, shots: 5
Virology Sesotho (st) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-difference, task: MMLU Virology, language: st, shots: 5
Virology Setswana (tn) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-difference, task: MMLU Virology, language: tn, shots: 5
Virology Setswana (tn) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-difference, task: MMLU Virology, language: tn, shots: 5
Virology Shona (sn) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-difference, task: MMLU Virology, language: sn, shots: 5
Virology Shona (sn) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-difference, task: MMLU Virology, language: sn, shots: 5
Virology Tsonga (ts) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-difference, task: MMLU Virology, language: ts, shots: 5
Virology Tsonga (ts) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-difference, task: MMLU Virology, language: ts, shots: 5
Virology Xhosa (xh) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-difference, task: MMLU Virology, language: xh, shots: 5
Virology Xhosa (xh) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-difference, task: MMLU Virology, language: xh, shots: 5
Virology Zulu (zu) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: back-translation-difference, task: MMLU Virology, language: zu, shots: 5
Virology Zulu (zu) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.14, section: machine-translation-difference, task: MMLU Virology, language: zu, shots: 5
Winogrande 11-language African average
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 1, section: average, task: Winogrande
Winogrande Afrikaans (af) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-accuracy, task: Winogrande, language: af, shots: 5
Winogrande Afrikaans (af) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: Winogrande, language: af, shots: 5
Winogrande Afrikaans (af) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-difference, task: Winogrande, language: af, shots: 5
Winogrande Afrikaans (af) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-difference, task: Winogrande, language: af, shots: 5
Winogrande Afrikaans (af) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-accuracy, task: Winogrande, language: af, shots: 5
Winogrande Amharic (am) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-accuracy, task: Winogrande, language: am, shots: 5
Winogrande Amharic (am) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: Winogrande, language: am, shots: 5
Winogrande Amharic (am) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-difference, task: Winogrande, language: am, shots: 5
Winogrande Amharic (am) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-difference, task: Winogrande, language: am, shots: 5
Winogrande Amharic (am) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-accuracy, task: Winogrande, language: am, shots: 5
Winogrande Bambara (bm) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-accuracy, task: Winogrande, language: bm, shots: 5
Winogrande Bambara (bm) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: Winogrande, language: bm, shots: 5
Winogrande Bambara (bm) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-difference, task: Winogrande, language: bm, shots: 5
Winogrande Bambara (bm) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-difference, task: Winogrande, language: bm, shots: 5
Winogrande Bambara (bm) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-accuracy, task: Winogrande, language: bm, shots: 5
Winogrande English (en) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-difference, task: Winogrande, language: en, shots: 5
Winogrande English (en) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-difference, task: Winogrande, language: en, shots: 5
Winogrande English (en) original
Source: Bridging the Gap (arXiv:2412.12417v1) data
By section · table: A.11 · task: Winogrande · language: en · shots: 5
table: 1, section: baseline, task: Winogrande
table: 2, task: Winogrande, language: en, shots: 5
Winogrande Igbo (ig) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-accuracy, task: Winogrande, language: ig, shots: 5
Winogrande Igbo (ig) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: Winogrande, language: ig, shots: 5
Winogrande Igbo (ig) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-difference, task: Winogrande, language: ig, shots: 5
Winogrande Igbo (ig) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-difference, task: Winogrande, language: ig, shots: 5
Winogrande Igbo (ig) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-accuracy, task: Winogrande, language: ig, shots: 5
Winogrande Sepedi (nso) cultural appropriateness target-language minus English lift (QA Pairs with Only “good” Quality Annotations)
Source: Bridging the Gap (arXiv:2412.12417v1) data
section: Results prose, table: A.19, row: G, statistic: maximum
table: A.19, section: G, language: nso
Winogrande Sepedi (nso) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-accuracy, task: Winogrande, language: nso, shots: 5
Winogrande Sepedi (nso) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: Winogrande, language: nso, shots: 5
Winogrande Sepedi (nso) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-difference, task: Winogrande, language: nso, shots: 5
Winogrande Sepedi (nso) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-difference, task: Winogrande, language: nso, shots: 5
Winogrande Sepedi (nso) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-accuracy, task: Winogrande, language: nso, shots: 5
Winogrande Sesotho (st) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-accuracy, task: Winogrande, language: st, shots: 5
Winogrande Sesotho (st) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: Winogrande, language: st, shots: 5
Winogrande Sesotho (st) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-difference, task: Winogrande, language: st, shots: 5
Winogrande Sesotho (st) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-difference, task: Winogrande, language: st, shots: 5
Winogrande Sesotho (st) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-accuracy, task: Winogrande, language: st, shots: 5
Winogrande Setswana (tn) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-accuracy, task: Winogrande, language: tn, shots: 5
Winogrande Setswana (tn) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: Winogrande, language: tn, shots: 5
Winogrande Setswana (tn) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-difference, task: Winogrande, language: tn, shots: 5
Winogrande Setswana (tn) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-difference, task: Winogrande, language: tn, shots: 5
Winogrande Setswana (tn) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-accuracy, task: Winogrande, language: tn, shots: 5
Winogrande Shona (sn) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-accuracy, task: Winogrande, language: sn, shots: 5
Winogrande Shona (sn) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: Winogrande, language: sn, shots: 5
Winogrande Shona (sn) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-difference, task: Winogrande, language: sn, shots: 5
Winogrande Shona (sn) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-difference, task: Winogrande, language: sn, shots: 5
Winogrande Shona (sn) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-accuracy, task: Winogrande, language: sn, shots: 5
Winogrande Tsonga (ts) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-accuracy, task: Winogrande, language: ts, shots: 5
Winogrande Tsonga (ts) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: Winogrande, language: ts, shots: 5
Winogrande Tsonga (ts) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-difference, task: Winogrande, language: ts, shots: 5
Winogrande Tsonga (ts) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-difference, task: Winogrande, language: ts, shots: 5
Winogrande Tsonga (ts) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-accuracy, task: Winogrande, language: ts, shots: 5
Winogrande Xhosa (xh) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-accuracy, task: Winogrande, language: xh, shots: 5
Winogrande Xhosa (xh) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: Winogrande, language: xh, shots: 5
Winogrande Xhosa (xh) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-difference, task: Winogrande, language: xh, shots: 5
Winogrande Xhosa (xh) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-difference, task: Winogrande, language: xh, shots: 5
Winogrande Xhosa (xh) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-accuracy, task: Winogrande, language: xh, shots: 5
Winogrande Zulu (zu) human translation back-translated to English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-accuracy, task: Winogrande, language: zu, shots: 5
Winogrande Zulu (zu) human-translated
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: 2, task: Winogrande, language: zu, shots: 5
Winogrande Zulu (zu) human-translation minus back-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: back-translation-difference, task: Winogrande, language: zu, shots: 5
Winogrande Zulu (zu) human-translation minus machine-translation difference
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-difference, task: Winogrande, language: zu, shots: 5
Winogrande Zulu (zu) machine-translated from English
Source: Bridging the Gap (arXiv:2412.12417v1) data
table: A.11, section: machine-translation-accuracy, task: Winogrande, language: zu, shots: 5
EKI EstonianSource: Keelemudelite mõõdupuu data docs
FormosanBench summarization12 related benchmarks · 12 results
ROUGE-2 (Amis, 10-shot)
Source: FormosanBench (arXiv:2506.21563v1) data
10-shot, 10 in-context examples, Mandarin prompt
ROUGE-2 (Atayal, 10-shot)
Source: FormosanBench (arXiv:2506.21563v1) data
10-shot, 10 in-context examples, Mandarin prompt
ROUGE-2 (Paiwan, 10-shot)
Source: FormosanBench (arXiv:2506.21563v1) data
10-shot, 10 in-context examples, Mandarin prompt
Show 6 more resultsHide 6 results
ROUGE-L (Amis, 10-shot)
Source: FormosanBench (arXiv:2506.21563v1) data
10-shot, 10 in-context examples, Mandarin prompt
ROUGE-L (Atayal, 10-shot)
Source: FormosanBench (arXiv:2506.21563v1) data
10-shot, 10 in-context examples, Mandarin prompt
ROUGE-L (Paiwan, 10-shot)
Source: FormosanBench (arXiv:2506.21563v1) data
10-shot, 10 in-context examples, Mandarin prompt
MMMLUSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
PashtoBench10 related benchmarks · 10 results
Show 4 more resultsHide 4 results
SeaLLMs16 related benchmarks · 16 results
Show 10 more resultsHide 10 results

Agentic

AgentDojo3 related benchmarks · 27 results
utility
Source: AgentDojo results data docs · Jun 2024
By attack · provider: openai · defense: None
By defense · provider: openai · attack: important_instructions
Targeted ASR
Source: AgentDojo results data docs · Jun 2024
By attack · provider: openai · defense: None
By defense · provider: openai · attack: important_instructions
utility under attack
Source: AgentDojo results data docs · Jun 2024
By attack · provider: openai · defense: None
By defense · provider: openai · attack: important_instructions
BrowserGymSource: BrowserGym Leaderboard
GAIA4 results
Source: GAIA Leaderboard data · Jun 2024
Difficulty level
Difficulty not reported
Gaia2Source: Gaia2 (Meta ARE) data · Sep 2025
Orak14 related benchmarks · 62 results
Source: Orak leaderboard docs
Orak 2048
AceAttorney
BabaIsYou
DarkestDungeon
HerStory
MineCraft
Show 32 more resultsHide 32 results
Pokémon
SlayTheSpire
StarCraftII
StarCraftII Elo
StardewValley
StreetFighterIII
StreetFighterIII Elo
Supermario
Tau2 Retail4 related benchmarks · 7 results
Tau2 Retail
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Tau2 Airline
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Tau2 Telecom
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
tau2-bench airline
Source: OpenRouter benchmark exports · Oct 2026
OpenRouter's automatic routing · gpt-4o-2024-08-06 · 90-day results
OpenAI · gpt-4o-2024-08-06 · 90-day results
OpenRouter's automatic routing · gpt-4o · 90-day results
OpenAI · gpt-4o · 90-day results

Image generation

UniGenBench++136 related benchmarks · 136 results
Source: UniGenBench++ data
BenchmarkScore
(Chinese) - 2D
(Chinese Long) - 2D
(Chinese Long) - 3D
(Chinese Long) - Action-Overall
Show 132 more benchmarksHide 132 benchmarks
BenchmarkScore
(Chinese Long) - Animal
(Chinese Long) - Attribute-Overall
(Chinese Long) - Color
(Chinese Long) - Comparison
(Chinese Long) - Composition
(Chinese Long) - Compound-Overall
(Chinese Long) - Consistency
(Chinese Long) - Contact
(Chinese Long) - Expression
(Chinese Long) - Feature matching
(Chinese Long) - Full body
(Chinese Long) - Grammar-Overall
(Chinese Long) - Hand
(Chinese Long) - Imagination
(Chinese Long) - Inclusion
(Chinese Long) - Layout-Overall
(Chinese Long) - Logical Reasoning
(Chinese Long) - Material
(Chinese Long) - Negation
(Chinese Long) - Non Contact
(Chinese Long) - Overall
(Chinese Long) - Pronoun Reference
(Chinese Long) - Quantity
(Chinese Long) - Relationship-Overall
(Chinese Long) - Shape
(Chinese Long) - Similarity
(Chinese Long) - Size
(Chinese Long) - State
(Chinese Long) - Style
(Chinese Long) - Text
(Chinese Long) - World Knowledge
(Chinese) - 3D
(Chinese) - Action-Overall
(Chinese) - Animal
(Chinese) - Attribute-Overall
(Chinese) - Color
(Chinese) - Comparison
(Chinese) - Composition
(Chinese) - Compound-Overall
(Chinese) - Consistency
(Chinese) - Contact
(Chinese) - Expression
(Chinese) - Feature matching
(Chinese) - Full body
(Chinese) - Grammar-Overall
(Chinese) - Hand
(Chinese) - Imagination
(Chinese) - Inclusion
(Chinese) - Layout-Overall
(Chinese) - Logical Reasoning
(Chinese) - Material
(Chinese) - Negation
(Chinese) - Non Contact
(Chinese) - Overall
(Chinese) - Pronoun Reference
(Chinese) - Quantity
(Chinese) - Relationship-Overall
(Chinese) - Shape
(Chinese) - Similarity
(Chinese) - Size
(Chinese) - State
(Chinese) - Style
(Chinese) - Text
(Chinese) - World Knowledge
(English Long) - 2D
(English Long) - 3D
(English Long) - Action-Overall
(English Long) - Animal
(English Long) - Attribute-Overall
(English Long) - Color
(English Long) - Comparison
(English Long) - Composition
(English Long) - Compound-Overall
(English Long) - Consistency
(English Long) - Contact
(English Long) - Expression
(English Long) - Feature matching
(English Long) - Full body
(English Long) - Grammar-Overall
(English Long) - Hand
(English Long) - Imagination
(English Long) - Inclusion
(English Long) - Layout-Overall
(English Long) - Logical Reasoning
(English Long) - Material
(English Long) - Negation
(English Long) - Non Contact
(English Long) - Overall
(English Long) - Pronoun Reference
(English Long) - Quantity
(English Long) - Relationship-Overall
(English Long) - Shape
(English Long) - Similarity
(English Long) - Size
(English Long) - State
(English Long) - Style
(English Long) - Text
(English Long) - World Knowledge
(English) - 2D
(English) - 3D
(English) - Action-Overall
(English) - Animal
(English) - Attribute-Overall
(English) - Color
(English) - Comparison
(English) - Composition
(English) - Compound-Overall
(English) - Consistency
(English) - Contact
(English) - Expression
(English) - Feature matching
(English) - Full body
(English) - Grammar-Overall
(English) - Hand
(English) - Imagination
(English) - Inclusion
(English) - Layout-Overall
(English) - Logical Reasoning
(English) - Material
(English) - Negation
(English) - Non Contact
(English) - Overall
(English) - Pronoun Reference
(English) - Quantity
(English) - Relationship-Overall
(English) - Shape
(English) - Similarity
(English) - Size
(English) - State
(English) - Style
(English) - Text
(English) - World Knowledge

Safety

HELM66 related benchmarks · 125 results
Show 114 more resultsHide 114 results
AIR-Bench: AIRBench 2024 - #13: Harassment
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
AIR-Bench: AIRBench 2024 - #18: Erotic
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
AIR-Bench: AIRBench 2024 - #20: Monetized
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
AIR-Bench: AIRBench 2024 - #32: Fraud
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
AIR-Bench: AIRBench 2024 - Child Harm
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
AIR-Bench: AIRBench 2024 - Deception
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
AIR-Bench: AIRBench 2024 - Defamation
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
AIR-Bench: AIRBench 2024 - Economic Harm
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
AIR-Bench: AIRBench 2024 - Hate/Toxicity
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
AIR-Bench: AIRBench 2024 - Manipulation
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
AIR-Bench: AIRBench 2024 - Political Usage
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
AIR-Bench: AIRBench 2024 - Privacy
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
AIR-Bench: AIRBench 2024 - Refusal Rate
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
AIR-Bench: AIRBench 2024 - Security Risks
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
AIR-Bench: AIRBench 2024 - Self-harm
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
AIR-Bench: AIRBench 2024 - Sexual Content
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
PhareSource: Phare

Knowledge

HELM60 related benchmarks · 116 results
Show 108 more resultsHide 108 results
MMLU: Astronomy - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: Business Ethics - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: Clinical Knowledge - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: College Biology - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: College Chemistry - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: College Computer Science - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: College Mathematics - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: College Medicine - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: College Physics - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: Computer Security - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: Conceptual Physics - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: Econometrics - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: Electrical Engineering - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: Elementary Mathematics - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: High School Biology - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: High School Chemistry - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: High School European History - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: High School Geography - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: High School Macroeconomics - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: High School Mathematics - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: High School Microeconomics - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: High School Physics - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: High School Psychology - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: High School Statistics - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: High School US History - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: High School World History - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: Human Aging - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: Human Sexuality - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: International Law - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: Jurisprudence - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: Logical Fallacies - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: Machine Learning - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: Management - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: Marketing - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: Medical Genetics - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: Miscellaneous - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: MMLU All Subjects - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: Moral Disputes - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: Moral Scenarios - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: Nutrition - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: Philosophy - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: Prehistory - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: Professional Accounting - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: Professional Law - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: Professional Medicine - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: Professional Psychology - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: Public Relations - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: Security Studies - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: Sociology - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: Us Foreign Policy - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: Virology - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU: World Religions - EM
Source: Stanford HELM specialized suites data docs
Setting not stated
Setting not stated
MMLU-ProSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim

Math

AIME 2024Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Source: JudgeBench data docs
Prompted Judge · Arena-Hard (gpt-4o-2024-05-13) · Math Score
GPT-4o Dataset, Multi-Agent Judge, ChatEval (gpt-4o-2024-05-13), Math Score
GPT-4o Dataset, Prompted Judge, Vanilla (gpt-4o-2024-05-13), Math Score
MATHSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
MathVistaSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
MGSMSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
mu-MATH7 related benchmarks · 35 results
F1
Source: U-MATH and mu-MATH
F1 · Qwen/Qwen2.5-72B-Instruct
accuracy
Source: U-MATH and mu-MATH
accuracy · Qwen/Qwen2.5-72B-Instruct
MCC
Source: U-MATH and mu-MATH
MCC · Qwen/Qwen2.5-72B-Instruct
NPV
Source: U-MATH and mu-MATH
NPV · Qwen/Qwen2.5-72B-Instruct
PPV
Source: U-MATH and mu-MATH
PPV · Qwen/Qwen2.5-72B-Instruct
TNR
Source: U-MATH and mu-MATH
TNR · Qwen/Qwen2.5-72B-Instruct
Show 5 more resultsHide 5 results
TPR
Source: U-MATH and mu-MATH
TPR · Qwen/Qwen2.5-72B-Instruct
Omni-MATH12 results
Source: Omni-MATH · May 2024
Setting not stated
Applied Mathematics
Show 6 more resultsHide 6 results
Discrete Mathematics
PutnamBench3 related benchmarks · 6 results
coq-wsolution solved
Source: PutnamBench
COPRA (GPT-4o), coq-wsolution, pass@1, pass@1 w/ T=0.0; 1988 B1.
GPT-4o, coq-wsolution, pass@10, pass@10 w/ T=0.7; 1988 B1.
isabelle-wsolution solved
Source: PutnamBench
DSP (GPT-4o), isabelle-wsolution, pass@10, pass@10 w/ T=0.1; 1971 B1, 1986 B1, 1995 B1, 2001 A1.
GPT-4o, isabelle-wsolution, pass@10, pass@10 w/ T=0.7; 1988 B1.
lean-wsolution solved
Source: PutnamBench
COPRA (GPT-4o), lean-wsolution, pass@1, pass@1 w/ T=0.0; 1988 B1.
GPT-4o, lean-wsolution, pass@10, pass@10 w/ T=0.7; 1988 B1.
U-MATH7 related benchmarks · 21 results
algebra accuracy
Source: U-MATH and mu-MATH
Task set · algebra · gpt-4o-2024-08-06
differential_calc accuracy
Source: U-MATH and mu-MATH
Task set · differential_calc · gpt-4o-2024-08-06
integral_calc accuracy
Source: U-MATH and mu-MATH
Task set · integral_calc · gpt-4o-2024-08-06
multivariable_calculus accuracy
Source: U-MATH and mu-MATH
Task set · multivariable_calculus · gpt-4o-2024-08-06
overall accuracy
Source: U-MATH and mu-MATH
Task set · u_math · gpt-4o-2024-08-06
precalculus_review accuracy
Source: U-MATH and mu-MATH
Task set · precalculus_review · gpt-4o-2024-08-06
Show 3 more resultsHide 3 results
sequences_series accuracy
Source: U-MATH and mu-MATH
Task set · sequences_series · gpt-4o-2024-08-06
ValsSource: Vals AI
WildBench5 related benchmarks · 5 results
WB_score.Math & Data Analysis
Source: WildBench direct data
allenai/WildBench, WB_score.Math & Data Analysis
gpt4t_reward.Math & Data Analysis.K=-1
Source: WildBench direct data
allenai/WildBench, gpt4t_reward.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09
haiku_reward.Math & Data Analysis.K=-1
Source: WildBench direct data
allenai/WildBench, haiku_reward.Math & Data Analysis.K=-1, claude-3-haiku-20240307
llama_reward.Math & Data Analysis.K=-1
Source: WildBench direct data
allenai/WildBench, llama_reward.Math & Data Analysis.K=-1, Llama-2-70b-chat-hf
mixture_of_rewards.Math & Data Analysis.K=-1
Source: WildBench direct data
allenai/WildBench, mixture_of_rewards.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf

Human preference

Arena ELO33 related benchmarks · 63 results
Source: LMArena data · Nov 2026
Arena ELO
Chinese
Creative Writing
English
Exclude Ties
Expert
Show 51 more resultsHide 51 results
French
German
Hard Prompts
Hard Prompts English
Industry Entertainment and Sports and Media
Industry Life and Physical and Social Science
Industry Mathematical
Industry Medicine and Healthcare
Industry Software and It Services
Industry Writing and Literature and Language
Japanese
Korean
Longer Query
Multi Turn
Non English
Polish
Russian
Spanish
Style-Controlled
Vision
Vision - Chinese
Vision - English
Vision Style-Controlled
WildBench26 related benchmarks · 26 results
WB_Elo
Source: WildBench direct data
allenai/WildBench, WB_Elo
gpt4t_reward.Creative Tasks.K=-1
Source: WildBench direct data
allenai/WildBench, gpt4t_reward.Creative Tasks.K: -1, gpt-4-turbo-2024-04-09
gpt4t_reward.Information/Advice seeking.K=-1
Source: WildBench direct data
allenai/WildBench, gpt4t_reward.Information/Advice seeking.K=-1, gpt-4-turbo-2024-04-09
gpt4t_reward.K=-1
Source: WildBench direct data
allenai/WildBench, gpt4t_reward.K: -1, gpt-4-turbo-2024-04-09
gpt4t_reward.Planning & Reasoning.K=-1
Source: WildBench direct data
allenai/WildBench, gpt4t_reward.Planning & Reasoning.K=-1, gpt-4-turbo-2024-04-09
gpt4t_reward.task_macro.K=-1
Source: WildBench direct data
allenai/WildBench, gpt4t_reward.task_macro.K: -1, gpt-4-turbo-2024-04-09
Show 20 more resultsHide 20 results
haiku_reward.Creative Tasks.K=-1
Source: WildBench direct data
allenai/WildBench, haiku_reward.Creative Tasks.K: -1, claude-3-haiku-20240307
haiku_reward.Information/Advice seeking.K=-1
Source: WildBench direct data
allenai/WildBench, haiku_reward.Information/Advice seeking.K=-1, claude-3-haiku-20240307
haiku_reward.K=-1
Source: WildBench direct data
allenai/WildBench, haiku_reward.K: -1, claude-3-haiku-20240307
haiku_reward.Planning & Reasoning.K=-1
Source: WildBench direct data
allenai/WildBench, haiku_reward.Planning & Reasoning.K=-1, claude-3-haiku-20240307
haiku_reward.task_macro.K=-1
Source: WildBench direct data
allenai/WildBench, haiku_reward.task_macro.K: -1, claude-3-haiku-20240307
llama_reward.Creative Tasks.K=-1
Source: WildBench direct data
allenai/WildBench, llama_reward.Creative Tasks.K: -1, Llama-2-70b-chat-hf
llama_reward.Information/Advice seeking.K=-1
Source: WildBench direct data
allenai/WildBench, llama_reward.Information/Advice seeking.K=-1, Llama-2-70b-chat-hf
llama_reward.K=-1
Source: WildBench direct data
allenai/WildBench, llama_reward.K: -1, Llama-2-70b-chat-hf
llama_reward.Planning & Reasoning.K=-1
Source: WildBench direct data
allenai/WildBench, llama_reward.Planning & Reasoning.K=-1, Llama-2-70b-chat-hf
llama_reward.task_macro.K=-1
Source: WildBench direct data
allenai/WildBench, llama_reward.task_macro.K: -1, Llama-2-70b-chat-hf
mixture_of_rewards.Creative Tasks.K=-1
Source: WildBench direct data
allenai/WildBench, mixture_of_rewards.Creative Tasks.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf
mixture_of_rewards.Information/Advice seeking.K=-1
Source: WildBench direct data
allenai/WildBench, mixture_of_rewards.Information/Advice seeking.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf
mixture_of_rewards.K=-1
Source: WildBench direct data
allenai/WildBench, mixture_of_rewards.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf
mixture_of_rewards.Planning & Reasoning.K=-1
Source: WildBench direct data
allenai/WildBench, mixture_of_rewards.Planning & Reasoning.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf
task_macro_reward.K=-1
Source: WildBench direct data
allenai/WildBench, task_macro_reward.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf
WB_score
Source: WildBench direct data
allenai/WildBench, WB_score
WB_score.Creative Tasks
Source: WildBench direct data
allenai/WildBench, WB_score.Creative Tasks
WB_score.Information/Advice seeking
Source: WildBench direct data
allenai/WildBench, WB_score.Information/Advice seeking
WB_score.Planning & Reasoning
Source: WildBench direct data
allenai/WildBench, WB_score.Planning & Reasoning
WB_score.task_macro
Source: WildBench direct data
allenai/WildBench, WB_score.task_macro

Coding

HumanEvalSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Source: JudgeBench data docs
Prompted Judge · Arena-Hard (gpt-4o-2024-05-13) · Coding Score
GPT-4o Dataset, Multi-Agent Judge, ChatEval (gpt-4o-2024-05-13), Coding Score
GPT-4o Dataset, Prompted Judge, Vanilla (gpt-4o-2024-05-13), Coding Score
LiveCodeBench8 related benchmarks · 8 results
AtCoder
Source: LiveCodeBench Leaderboard data
Codeforces
Source: LiveCodeBench Leaderboard data
platform:codeforces
Easy
Source: LiveCodeBench Leaderboard data
Hard
Source: LiveCodeBench Leaderboard data
LeetCode
Source: LiveCodeBench Leaderboard data
Show 2 more resultsHide 2 results
Medium
Source: LiveCodeBench Leaderboard data
Pro
Source: LiveCodeBench Pro data · Oct 2025
Multi-SWE-bench9 related benchmarks · 27 results
C Verified
Source: Multi-SWE-bench Experiments data docs · Mar 2025
MagentLess, 20250329_MagentLess_GPT-4o-1120
MopenHands, 20250329_MopenHands_GPT-4o-1120
MSWE-agent, 20250329_MSWE-agent_GPT-4o-1120
C++ Verified
Source: Multi-SWE-bench Experiments data docs · Mar 2025
MagentLess, 20250329_MagentLess_GPT-4o-1120
MopenHands, 20250329_MopenHands_GPT-4o-1120
MSWE-agent, 20250329_MSWE-agent_GPT-4o-1120
Go Verified
Source: Multi-SWE-bench Experiments data docs · Mar 2025
MagentLess, 20250329_MagentLess_GPT-4o-1120
MopenHands, 20250329_MopenHands_GPT-4o-1120
MSWE-agent, 20250329_MSWE-agent_GPT-4o-1120
Java Verified
Source: Multi-SWE-bench Experiments data docs · Mar 2025
MagentLess, 20250329_MagentLess_GPT-4o-1120
MopenHands, 20250329_MopenHands_GPT-4o-1120
MSWE-agent, 20250329_MSWE-agent_GPT-4o-1120
JavaScript Verified
Source: Multi-SWE-bench Experiments data docs · Mar 2025
MagentLess, 20250329_MagentLess_GPT-4o-1120
MopenHands, 20250329_MopenHands_GPT-4o-1120
MSWE-agent, 20250329_MSWE-agent_GPT-4o-1120
Lite All Verified
Source: Multi-SWE-bench Experiments data docs · Mar 2025
MagentLess, 20250329_MagentLess_GPT-4o-1120
MopenHands, 20250329_MopenHands_GPT-4o-1120
MSWE-agent, 20250329_MSWE-agent_GPT-4o-1120
Show 9 more resultsHide 9 results
Python Verified
Source: Multi-SWE-bench Experiments data docs · Mar 2025
MagentLess, 20250329_Agentless_GPT-4o-1120
MopenHands, 20250329_OpenHands_GPT-4o-1120
MSWE-agent, 20250329_SWE-agent_GPT-4o-1120
Rust Verified
Source: Multi-SWE-bench Experiments data docs · Mar 2025
MagentLess, 20250329_MagentLess_GPT-4o-1120
MopenHands, 20250329_MopenHands_GPT-4o-1120
MSWE-agent, 20250329_MSWE-agent_GPT-4o-1120
TypeScript Verified
Source: Multi-SWE-bench Experiments data docs · Mar 2025
MagentLess, 20250329_MagentLess_GPT-4o-1120
MopenHands, 20250329_MopenHands_GPT-4o-1120
MSWE-agent, 20250329_MSWE-agent_GPT-4o-1120
SWE-Lancer8 related benchmarks · 10 results
SWE-Lancer
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
(IC-Diamond subset)
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
SWE-bench Verified
Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Show 4 more resultsHide 4 results
SWE-bench-Live Full
Source: SWE-bench-Live leaderboard data · May 2025
scaffold: swe-agent, set: full, instances: 1319
SWE-bench-Live Lite
Source: SWE-bench-Live leaderboard data · May 2025
By scaffold · set: lite · instances: 300
WildBench5 related benchmarks · 5 results
WB_score.Coding & Debugging
Source: WildBench direct data
allenai/WildBench, WB_score.Coding & Debugging
gpt4t_reward.Coding & Debugging.K=-1
Source: WildBench direct data
allenai/WildBench, gpt4t_reward.Coding & Debugging.K=-1, gpt-4-turbo-2024-04-09
haiku_reward.Coding & Debugging.K=-1
Source: WildBench direct data
allenai/WildBench, haiku_reward.Coding & Debugging.K=-1, claude-3-haiku-20240307
llama_reward.Coding & Debugging.K=-1
Source: WildBench direct data
allenai/WildBench, llama_reward.Coding & Debugging.K=-1, Llama-2-70b-chat-hf
mixture_of_rewards.Coding & Debugging.K=-1
Source: WildBench direct data
allenai/WildBench, mixture_of_rewards.Coding & Debugging.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf

Vision

AI2DSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
ChartQASource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
CharXiv-D13 related benchmarks · 13 results
Show 7 more resultsHide 7 results
DocVQASource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
MMMUSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
MMMU-ProSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
VideoMMMUSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim

Science

GPQASource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
GPQA Diamond7 results
Source: Epoch AI Benchmarking Hub data
Setting not stated
Source: OpenRouter benchmark exports · Oct 2026
gpt-4o-2024-08-06 · 90-day results
Across 2 providers lowest 51.5% median 52.2% highest 52.8%OpenRouter's automatic routing 52%
Show results from 2 providers and OpenRouter's automatic routingHide results from 2 providers and OpenRouter's automatic routing
  • OpenAI
  • OpenRouter's automatic routing
  • Azure
gpt-4o · 90-day results
Across 2 providers lowest 51.7% median 52.2% highest 52.7%OpenRouter's automatic routing 51.1%
Show results from 2 providers and OpenRouter's automatic routingHide results from 2 providers and OpenRouter's automatic routing
  • OpenAI
  • Azure
  • OpenRouter's automatic routing

Reasoning

COLLIESource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
DROPSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
EgoSchemaSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
ERQASource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Source: JudgeBench data docs
Prompted Judge · Arena-Hard (gpt-4o-2024-05-13) · Reasoning Score
GPT-4o Dataset, Multi-Agent Judge, ChatEval (gpt-4o-2024-05-13), Reasoning Score
GPT-4o Dataset, Prompted Judge, Vanilla (gpt-4o-2024-05-13), Reasoning Score

Instruction following

IFEvalSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Multi-IFSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim

General

Factuality

SimpleQASource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim

OCR

Speech

Long context

Other

Horangi36 related benchmarks · 36 results
Source: JudgeBench data docs
Prompted Judge · Arena-Hard (gpt-4o-2024-05-13) · Knowledge Score
GPT-4o Dataset, Multi-Agent Judge, ChatEval (gpt-4o-2024-05-13), Knowledge Score
GPT-4o Dataset, Prompted Judge, Vanilla (gpt-4o-2024-05-13), Knowledge Score
Source: JudgeBench data docs
Prompted Judge · Arena-Hard (gpt-4o-2024-05-13) · Overall Score
GPT-4o Dataset, Multi-Agent Judge, ChatEval (gpt-4o-2024-05-13), Overall Score
GPT-4o Dataset, Prompted Judge, Vanilla (gpt-4o-2024-05-13), Overall Score
MedHELM33 related benchmarks · 33 results
Nejumi 428 related benchmarks · 28 results
Open Greek FinLLM11 related benchmarks · 11 results
QA raw score
Source: TheFinAI public numeric results
greek-contents, QA, QA Raw
FinNum raw score
Source: TheFinAI public numeric results
greek-contents, FinNum, FinNum Raw
FinText raw score
Source: TheFinAI public numeric results
greek-contents, FinText, FinText Raw
Show 5 more resultsHide 5 results
FNS raw score
Source: TheFinAI public numeric results
greek-contents, FNS, FNS Raw
MultiFin raw score
Source: TheFinAI public numeric results
greek-contents, MultiFin, MultiFin Raw
overall normalized index
Source: TheFinAI public numeric results
greek-contents, published-average
ValsSource: Vals AI

Data credits: ACEBench, MIT (licence) · Data by LLM Stats · AgentDojo. Copyright (c) 2024 Edoardo Debenedetti, Jie Zhang, Mislav Balunovic, Luca Beurer-Kellner, Marc Fischer, and Florian Tramèr. MIT License. Not a leaderboard: attack and defense coverage differs between models. (licence) · Aider LLM Leaderboards (Aider-AI), Apache-2.0 (licence) · Epoch AI Benchmarking Hub, CC BY 4.0 (licence) · LMArena, CC BY 4.0 (licence) · ArtifactsBench (Tencent Hunyuan), CC BY 4.0 (licence) · FutureHouse and ScienceMachine, BixBench v1.5 public zero-shot baseline file. Apache-2.0 repository. (licence) · Tuka Alhanai, Adam Kasumovic, Mohammad Ghassemi, Aven Zitzelberger, Jessica Lundin and Guillaume Chabot-Couture. Bridging the Gap: Enhancing LLM Performance for Low-Resource African Languages with New Benchmarks, Fine-Tuning, and Cultural Adjustments, arXiv:2412.12417v1. CC BY 4.0 article numeric/protocol extract, restructured with values unchanged. No task or model rights asserted. (licence) · Steel.dev Agent Leaderboards, MIT (licence) · BrowserGym Leaderboard (ServiceNow), MIT (licence) · BullshitBench by Peter Gostev, MIT (licence) · CharXiv (Wang et al., NeurIPS 2024), Princeton NLP. https://charxiv.github.io · ChemBench leaderboard, Jablonka group / lamalab-org. Public verified numeric result summaries; results licence not separately stated. · CSEBenchmark (Wang et al., IEEE S&P 2025). Copyright (c) 2024 NASP. MIT License. (licence) · CyBench — cite Zhang et al. 2025 · DABstep (Adyen), CC BY 4.0 (licence) · Keelemudelite mõõdupuu: Independent LLM leaderboard for Estonian. Eesti Keele Instituut. https://moodupuu.eki.ee, CC BY 4.0. Latest per-benchmark results selected with partial-run flags retained. (licence) · EvalPlus Leaderboard and EvalPerf (evalplus.github.io), Apache-2.0; EvalPerf win-rate fractions converted to percentages (licence) · FaithJudge, Vectara; Tamber et al., EMNLP 2025 Industry. Public numeric README leaderboard; results licence not separately established. · Lin, Chen and Zhang, FormosanBench: Benchmarking Low-Resource Austronesian Languages in the Era of Large Language Models, arXiv:2506.21563v1, CC BY 4.0. Scores extracted from Tables 2-4. (licence) · OpenRouter public benchmark leaderboard and search-lane exports, CC BY 4.0. Fractional source scores converted to percentages; immutable dated snapshots. (licence) · Stanford HELM (Stanford CRFM); scenario scores from Capabilities, Safety, Long Context, MedHELM, AIR-Bench, MMLU and Finance · Stanford HELM (Stanford CRFM), Apache-2.0 (licence) · Horangi 4, Weights & Biases Korea; public publisher-selected numeric evaluation results · JudgeBench, Sijun Tan et al., ScalerLab · LLM Chess by Maxim Saplin, Apache-2.0 (Copyright [2024] [Maxim Saplin]) (licence) · M-IFEval: Dussolle, Cardeña Díaz, Sato and Devine (Lightblue), arXiv:2502.04688. README leaderboard, repository licensed Apache-2.0. (licence) · MathArena (ETH SRI / INSAIT), MIT — cite dekoninck2026matharena (licence) · MineBench, MIT (licence) · MLE-bench (OpenAI), MIT (licence) · Toloka, U-MATH and mu-MATH public leaderboard. MIT Space; generated solution outputs retain underlying model terms and are not ingested. (licence) · Multi-SWE-bench experiments (multi-swe-bench/experiments), Apache-2.0; resolved-rate fractions converted to percentages (licence) · Nejumi Leaderboard 4, Weights & Biases Japan; public publisher-selected numeric evaluation results · Online-Mind2Web Leaderboard (OSU NLP), Apache-2.0 (licence) · The Fin AI. Published FinBen result summaries and Open Greek Financial LLM Leaderboard aggregates. Results licence not separately stated; fractional FinBen success metrics converted to percentages. · Orak leaderboard, KRAFTON AI (Park et al., NeurIPS 2025). Public numeric scores; leaderboard licence not stated (code MIT not applied; CC BY-NC trajectories not used). · PashtoBench v1.0, Jawad Noori. Public results/scores numbers; code MIT and data/ CC BY 4.0 grants do not name results/, so no results licence is asserted. · pfgen-bench, Preferred Networks. Apache-2.0 except LLM-generated outputs; aggregate scores only. (licence) · Phare (Giskard), CC BY 4.0 (licence) · Physical AI Bench (SHI Labs), MIT (licence) · Haq et al., PsOCR: Benchmarking Large Multimodal Models for Optical Character Recognition in Low-resource Pashto Language, arXiv:2505.10055v2, CC BY 4.0. Scores extracted from Tables 3-5. (licence) · Trishul Lab, PutnamBench public submitted results. Model identities and scaffold budgets preserved; result licence unstated. · ScreenSpot grounding leaderboard (GUI-Agent), Apache-2.0 (licence) · SeaExam and SeaBench leaderboard, SeaLLMs (DAMO-NLP-SG). Published leaderboard scores; results licence not stated. · Shaberi v2.1, Shisa.AI, Apache-2.0. GPT-5.1-judge scores only; judges are not comparable. (licence) · SnakeBench (Greg Kamradt / ARC Prize Foundation), MIT (licence) · Swallow LLM Leaderboard v2, Swallow LLM team (Institute of Science Tokyo). Contents CC BY 4.0; Japanese post-trained task scores on the source's 0..1 scale. (licence) · SWE-bench-Live (Microsoft and contributors), swe-bench-live.github.io. Resolve rates computed from the published report counts. · SWE-Lancer community leaderboard. Offline task subset as sized in the publisher README; aggregate accuracy and earned payouts. · UniGenBench++ (CodeGoat24), MIT (licence) · UzLiB, Tahrirchi (Abror Shopulatov). Public LEADERBOARD.md numbers; repository MIT covers software and documentation, no separate results licence stated and none asserted. · Vectara Hallucination Leaderboard, Apache-2.0 (licence) · VideoGameBench, Alex L. Zhang, Thomas L. Griffiths, Karthik R. Narasimhan, Ofir Press. Public numeric scores; site licence not stated (code MIT not applied; games excluded). · WebWalkerQA (Alibaba NLP), Apache-2.0 (licence) · WildBench, Allen Institute for AI · WildVision Arena, MIT (licence)

Also known as: azure/eu/gpt-4o, azure/eu/gpt-4o-2024-05-13, azure/eu/gpt-4o-2024-08-06, azure/eu/gpt-4o-2024-11-20, azure/global-standard/gpt-4o-2024-08-06, azure/global-standard/gpt-4o-2024-11-20, azure/global/gpt-4o-2024-08-06, azure/global/gpt-4o-2024-11-20, azure/gpt-4o, azure/gpt-4o-2024-05-13, azure/gpt-4o-2024-08-06, azure/gpt-4o-2024-11-20, azure/us-gov/gpt-4o, azure/us-gov/gpt-4o-2024-05-13, azure/us-gov/gpt-4o-2024-11-20, azure/us/gpt-4o, azure/us/gpt-4o-2024-05-13, azure/us/gpt-4o-2024-08-06, azure/us/gpt-4o-2024-11-20, github_copilot/gpt-4o, github_copilot/gpt-4o-2024-05-13, github_copilot/gpt-4o-2024-08-06, github_copilot/gpt-4o-2024-11-20, gmi/openai/gpt-4o, gpt-4o-0513, gpt-4o-0806, gpt-4o-1120, gpt-4o-2024-05-13, gpt-4o-2024-08-06, gpt-4o-2024-11-20, gpt-4o-latest, gpt-4o-search, gpt-4o:extended, gradient_ai/openai-gpt-4o, openai-gpt-4o, openai-gpt-4o-2024-11-20, openai/gpt-4o, openai/gpt-4o-2024-05-13, openai/gpt-4o-2024-08-06, openai/gpt-4o-2024-11-20, openai/gpt-4o-batch, openai/gpt-4o-fast, openai/gpt-4o-search, openai/gpt-4o:batch, openai/gpt-4o:extended, replicate/openai/gpt-4o, vercel_ai_gateway/openai/gpt-4o

Like this one. Tap or hover any dotted value, label or icon to see where it comes from and how it is measured. open details when tapped or hovered. How this data works