Skip to content

Google · Chat

Gemma 2 9B IT

Google 🇺🇸ChatGemini familyOne of its sources says the weights aren't publicly available.
llminfo IQ
102.3
Market median price
$0.08 blended / 1M
Max context
8K
Listings
8

Evidence and sources

🤗 google/gemma-2-9b-itllminfo IQ 102.3, ranked among chat models. From 133 benchmarks. Plenty of data behind it.How llminfo IQ is computedCoding IQ 75.7, 95% interval 65–86. From 4 benchmark families across 3 sources. Little data behind it, so treat it as a rough estimate. Uses the model's best published results. It has its own scale, separate from llminfo IQ.How Coding IQ is computedReleased 27 Jun 2024Maker HQ: US (according to Google's own website)
Sources: LMArena data, Models.dev data, llminfo researchPrimary data from LMArena from Models.dev: weights availability from llminfo research: release date

Advantages and trade-offs

Compared with 10 chat models, picked mainly by popularity and IQ

Trade-offs

No open weights

The weights aren't public, so you can't run it yourself.

We only make a claim when there's enough data on models of the same type. Updated 22 Nov, 06:07 UTC.Prices come from the model's maker where they sell it, otherwise their official partner, otherwise the market median. This comparison uses 7 maker's prices; 3 market median prices.

Input & output

as stated by the model's providers
Text
Text

Listings

 Hugging FaceInference by Nebius AI Studiogoogle/gemma-2-9b-itSource: GenAI Prices data8K$0.03$0.09
 OpenRoutergoogle/gemma-2-9b-it:free · freeSource: Helicone Costs dataNot published by the provider$0.00$0.00
 Fireworks AIfireworks_ai/accounts/fireworks/models/gemma2-9b-itSource: LiteLLM data8K$0.20$0.20
 Groqgemma2-9b-itSource: GenAI Prices dataNot published by the provider$0.20$0.20
 Novita AIgoogle/gemma-2-9b-itSource: GenAI Prices dataNot published by the provider$0.08$0.08

3 listings without a price are hidden.

Benchmarks

Named effort levels run from lowest to highest. Select a result to see its source details.

Human preference

Arena ELO26 related benchmarks · 26 results
WildBench6 related benchmarks · 6 results
WB_score
Source: WildBench direct data
allenai/WildBench, WB_score
WB_Elo
Source: WildBench direct data
allenai/WildBench, WB_Elo
WB_score.Creative Tasks
Source: WildBench direct data
allenai/WildBench, WB_score.Creative Tasks
WB_score.Information/Advice seeking
Source: WildBench direct data
allenai/WildBench, WB_score.Information/Advice seeking
WB_score.Planning & Reasoning
Source: WildBench direct data
allenai/WildBench, WB_score.Planning & Reasoning
WB_score.task_macro
Source: WildBench direct data
allenai/WildBench, WB_score.task_macro

Science

Multilingual

Reasoning

ARC-CSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
ARC-ESource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
BoolQSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
HellaSwagSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
PIQASource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim

General

Math

AGIEvalSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
BIG-BenchSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
GSM8KSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
MATHSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim

Coding

HumanEvalSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim
MBPPSource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim

Knowledge

TriviaQASource: LLM Stats data · Reported by the model's developer. The source relaying it did not verify it. claim

Instruction following

Vision

Other

Open Greek FinLLM11 related benchmarks · 11 results
QA raw score
Source: TheFinAI public numeric results
greek-contents, QA, QA Raw
FinNum raw score
Source: TheFinAI public numeric results
greek-contents, FinNum, FinNum Raw
FinText raw score
Source: TheFinAI public numeric results
greek-contents, FinText, FinText Raw
Show 5 more resultsHide 5 results
FNS raw score
Source: TheFinAI public numeric results
greek-contents, FNS, FNS Raw
MultiFin raw score
Source: TheFinAI public numeric results
greek-contents, MultiFin, MultiFin Raw
overall normalized index
Source: TheFinAI public numeric results
greek-contents, published-average

Data credits: Data by LLM Stats · LMArena, CC BY 4.0 (licence) · ChemBench leaderboard, Jablonka group / lamalab-org. Public verified numeric result summaries; results licence not separately stated. · EQ-Bench (Samuel J. Paech) — cite paech2023eqbench · EvalPlus Leaderboard and EvalPerf (evalplus.github.io), Apache-2.0; EvalPerf win-rate fractions converted to percentages (licence) · Epoch AI Benchmarking Hub, CC BY 4.0 (licence) · Stanford HELM (Stanford CRFM), Apache-2.0 (licence) · LLM Chess by Maxim Saplin, Apache-2.0 (Copyright [2024] [Maxim Saplin]) (licence) · Open Arabic LLM Leaderboard (OALL), Apache-2.0 (licence) · The Fin AI. Published FinBen result summaries and Open Greek Financial LLM Leaderboard aggregates. Results licence not separately stated; fractional FinBen success metrics converted to percentages. · pfgen-bench, Preferred Networks. Apache-2.0 except LLM-generated outputs; aggregate scores only. (licence) · SeaExam and SeaBench leaderboard, SeaLLMs (DAMO-NLP-SG). Published leaderboard scores; results licence not stated. · WildBench, Allen Institute for AI

Also known as: fireworks_ai/accounts/fireworks/models/gemma2-9b-it, gemma-2-9b-it, google/gemma-2-9b-it, google/gemma-2-9b-it:free

Like this one. Tap or hover any dotted value, label or icon to see where it comes from and how it is measured. open details when tapped or hovered. How this data works