Skip to content

Google · Chat

Gemini 1.5 Pro

Google 🇺🇸ChatGemini family

Newer version: Gemini 2.5 Pro. We compared release dates and version names in our catalogue.

llminfo IQ
114.0
Max context
2M
Listings
3

Evidence and sources

llminfo IQ 114.0, ranked among chat models. From 156 benchmarks. Plenty of data behind it.How llminfo IQ is computedCoding IQ 80.0, 95% interval 69–92. From 3 benchmark families across 2 sources. Little data behind it, so treat it as a rough estimate. Uses the model's best published results. It has its own scale, separate from llminfo IQ.How Coding IQ is computedReleased 15 Feb 2024Maker HQ: US (according to Google's own website)
Sources: LMArena data, llminfo researchPrimary data from LMArena from llminfo research: release date

Advantages and trade-offs

Compared with 10 chat models, picked mainly by popularity and IQ

Advantages

We only make a claim when there's enough data on models of the same type. Updated 22 Nov, 04:34 UTC.Prices come from the model's maker where they sell it, otherwise their official partner, otherwise the market median. This comparison uses 7 maker's prices; 3 market median prices.

Input & output

from the model's type
Text
Text

Listings

 Googlegemini-1.5-proSource: GenAI Prices data1M$1.25>128k $2.50$5.00>128k $10.00
 302aigemini-1.5-pro-0801 · 0801Source: 302.ai data128K$3.50$10.50
 302aigemini-1.5-proSource: 302.ai data2M$7.00$21.00

Benchmarks

Named effort levels run from lowest to highest. Select a result to see its source details.

Human preference

Arena ELO31 related benchmarks · 61 results
Source: LMArena data · Nov 2026
Benchmark001002
Arena ELO
Chinese
Creative Writing
English
Show 27 more benchmarksHide 27 benchmarks
Benchmark001002
Exclude Ties
Expert
French
German
Hard Prompts
Hard Prompts English
Industry Business and Management and Financial Operations
Industry Entertainment and Sports and Media
Industry Life and Physical and Social Science
Industry Mathematical
Industry Medicine and Healthcare
Industry Software and It Services
Industry Writing and Literature and Language
Japanese
Korean
Longer Query
Multi Turn
Non English
PolishNot measured
Russian
Spanish
Style-Controlled
Vision
Vision - Chinese
Vision - English
Vision Style-Controlled
WildBench26 related benchmarks · 26 results
WB_Elo
Source: WildBench direct data
allenai/WildBench, WB_Elo
gpt4t_reward.Creative Tasks.K=-1
Source: WildBench direct data
allenai/WildBench, gpt4t_reward.Creative Tasks.K: -1, gpt-4-turbo-2024-04-09
gpt4t_reward.Information/Advice seeking.K=-1
Source: WildBench direct data
allenai/WildBench, gpt4t_reward.Information/Advice seeking.K=-1, gpt-4-turbo-2024-04-09
gpt4t_reward.K=-1
Source: WildBench direct data
allenai/WildBench, gpt4t_reward.K: -1, gpt-4-turbo-2024-04-09
gpt4t_reward.Planning & Reasoning.K=-1
Source: WildBench direct data
allenai/WildBench, gpt4t_reward.Planning & Reasoning.K=-1, gpt-4-turbo-2024-04-09
gpt4t_reward.task_macro.K=-1
Source: WildBench direct data
allenai/WildBench, gpt4t_reward.task_macro.K: -1, gpt-4-turbo-2024-04-09
Show 20 more resultsHide 20 results
haiku_reward.Creative Tasks.K=-1
Source: WildBench direct data
allenai/WildBench, haiku_reward.Creative Tasks.K: -1, claude-3-haiku-20240307
haiku_reward.Information/Advice seeking.K=-1
Source: WildBench direct data
allenai/WildBench, haiku_reward.Information/Advice seeking.K=-1, claude-3-haiku-20240307
haiku_reward.K=-1
Source: WildBench direct data
allenai/WildBench, haiku_reward.K: -1, claude-3-haiku-20240307
haiku_reward.Planning & Reasoning.K=-1
Source: WildBench direct data
allenai/WildBench, haiku_reward.Planning & Reasoning.K=-1, claude-3-haiku-20240307
haiku_reward.task_macro.K=-1
Source: WildBench direct data
allenai/WildBench, haiku_reward.task_macro.K: -1, claude-3-haiku-20240307
llama_reward.Creative Tasks.K=-1
Source: WildBench direct data
allenai/WildBench, llama_reward.Creative Tasks.K: -1, Llama-2-70b-chat-hf
llama_reward.Information/Advice seeking.K=-1
Source: WildBench direct data
allenai/WildBench, llama_reward.Information/Advice seeking.K=-1, Llama-2-70b-chat-hf
llama_reward.K=-1
Source: WildBench direct data
allenai/WildBench, llama_reward.K: -1, Llama-2-70b-chat-hf
llama_reward.Planning & Reasoning.K=-1
Source: WildBench direct data
allenai/WildBench, llama_reward.Planning & Reasoning.K=-1, Llama-2-70b-chat-hf
llama_reward.task_macro.K=-1
Source: WildBench direct data
allenai/WildBench, llama_reward.task_macro.K: -1, Llama-2-70b-chat-hf
mixture_of_rewards.Creative Tasks.K=-1
Source: WildBench direct data
allenai/WildBench, mixture_of_rewards.Creative Tasks.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf
mixture_of_rewards.Information/Advice seeking.K=-1
Source: WildBench direct data
allenai/WildBench, mixture_of_rewards.Information/Advice seeking.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf
mixture_of_rewards.K=-1
Source: WildBench direct data
allenai/WildBench, mixture_of_rewards.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf
mixture_of_rewards.Planning & Reasoning.K=-1
Source: WildBench direct data
allenai/WildBench, mixture_of_rewards.Planning & Reasoning.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf
task_macro_reward.K=-1
Source: WildBench direct data
allenai/WildBench, task_macro_reward.K: -1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf
WB_score
Source: WildBench direct data
allenai/WildBench, WB_score
WB_score.Creative Tasks
Source: WildBench direct data
allenai/WildBench, WB_score.Creative Tasks
WB_score.Information/Advice seeking
Source: WildBench direct data
allenai/WildBench, WB_score.Information/Advice seeking
WB_score.Planning & Reasoning
Source: WildBench direct data
allenai/WildBench, WB_score.Planning & Reasoning
WB_score.task_macro
Source: WildBench direct data
allenai/WildBench, WB_score.task_macro

Math

GSM8KSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
MATHSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
MathVistaSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
MGSMSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
mu-MATH7 related benchmarks · 35 results
F1
Source: U-MATH and mu-MATH
F1 · Qwen/Qwen2.5-72B-Instruct
accuracy
Source: U-MATH and mu-MATH
accuracy · Qwen/Qwen2.5-72B-Instruct
MCC
Source: U-MATH and mu-MATH
MCC · Qwen/Qwen2.5-72B-Instruct
NPV
Source: U-MATH and mu-MATH
NPV · Qwen/Qwen2.5-72B-Instruct
PPV
Source: U-MATH and mu-MATH
PPV · Qwen/Qwen2.5-72B-Instruct
TNR
Source: U-MATH and mu-MATH
TNR · Qwen/Qwen2.5-72B-Instruct
Show 5 more resultsHide 5 results
TPR
Source: U-MATH and mu-MATH
TPR · Qwen/Qwen2.5-72B-Instruct
U-MATH7 related benchmarks · 21 results
algebra accuracy
Source: U-MATH and mu-MATH
Task set · algebra · gpt-4o-2024-08-06
differential_calc accuracy
Source: U-MATH and mu-MATH
Task set · differential_calc · gpt-4o-2024-08-06
integral_calc accuracy
Source: U-MATH and mu-MATH
Task set · integral_calc · gpt-4o-2024-08-06
multivariable_calculus accuracy
Source: U-MATH and mu-MATH
Task set · multivariable_calculus · gpt-4o-2024-08-06
overall accuracy
Source: U-MATH and mu-MATH
Task set · u_math · gpt-4o-2024-08-06
precalculus_review accuracy
Source: U-MATH and mu-MATH
Task set · precalculus_review · gpt-4o-2024-08-06
Show 3 more resultsHide 3 results
sequences_series accuracy
Source: U-MATH and mu-MATH
Task set · sequences_series · gpt-4o-2024-08-06
WildBench5 related benchmarks · 5 results
WB_score.Math & Data Analysis
Source: WildBench direct data
allenai/WildBench, WB_score.Math & Data Analysis
gpt4t_reward.Math & Data Analysis.K=-1
Source: WildBench direct data
allenai/WildBench, gpt4t_reward.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09
haiku_reward.Math & Data Analysis.K=-1
Source: WildBench direct data
allenai/WildBench, haiku_reward.Math & Data Analysis.K=-1, claude-3-haiku-20240307
llama_reward.Math & Data Analysis.K=-1
Source: WildBench direct data
allenai/WildBench, llama_reward.Math & Data Analysis.K=-1, Llama-2-70b-chat-hf
mixture_of_rewards.Math & Data Analysis.K=-1
Source: WildBench direct data
allenai/WildBench, mixture_of_rewards.Math & Data Analysis.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf

Vision

MMMUSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Vibe-EvalSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
Video-MMESource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim

Coding

HumanEvalSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
WildBench5 related benchmarks · 5 results
WB_score.Coding & Debugging
Source: WildBench direct data
allenai/WildBench, WB_score.Coding & Debugging
gpt4t_reward.Coding & Debugging.K=-1
Source: WildBench direct data
allenai/WildBench, gpt4t_reward.Coding & Debugging.K=-1, gpt-4-turbo-2024-04-09
haiku_reward.Coding & Debugging.K=-1
Source: WildBench direct data
allenai/WildBench, haiku_reward.Coding & Debugging.K=-1, claude-3-haiku-20240307
llama_reward.Coding & Debugging.K=-1
Source: WildBench direct data
allenai/WildBench, llama_reward.Coding & Debugging.K=-1, Llama-2-70b-chat-hf
mixture_of_rewards.Coding & Debugging.K=-1
Source: WildBench direct data
allenai/WildBench, mixture_of_rewards.Coding & Debugging.K=-1, gpt-4-turbo-2024-04-09, claude-3-haiku-20240307, Llama-2-70b-chat-hf

Reasoning

DROPSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
HellaSwagSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
MMSUSource: MMSU data
MRCRSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim

Long context

Speech

FLEURSSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
MMSUSource: MMSU data

Science

GPQASource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim

Instruction following

Knowledge

MMLUSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
MMLU-ProSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim

Safety

XSTestSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim

Agentic

Factuality

Multilingual

WMT23Source: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim

OCR

Other

Data credits: OpenVLM Leaderboard (OpenCompass / VLMEvalKit), Apache-2.0 (licence) · ACEBench, MIT (licence) · Data by LLM Stats · LMArena, CC BY 4.0 (licence) · CharXiv (Wang et al., NeurIPS 2024), Princeton NLP. https://charxiv.github.io · CyBench — cite Zhang et al. 2025 · Toloka, U-MATH and mu-MATH public leaderboard. MIT Space; generated solution outputs retain underlying model terms and are not ingested. (licence) · Phare (Giskard), CC BY 4.0 (licence) · NVIDIA RULER, Hsieh et al. Official README numeric table; no separate results licence asserted. · WildBench, Allen Institute for AI

Also known as: gemini-1.5-pro-0801

Like this one. Tap or hover any dotted value, label or icon to see where it comes from and how it is measured. open details when tapped or hovered. How this data works