Skip to content

Benchmark

MTEB(fra, v1) - Summarization

Embedding benchmarks measure how well models find related items, for example when matching a search to relevant results. Scores are percentages.

Models measured
31
Scores recorded
31
Sources
1
IQ weight
0.39

Score distribution

30.1%
Mean
30.4%
Median
1.2%
Std dev
28.3%
10th percentile
31.3%
90th percentile
26.6%
Min
26.6%31.9%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.39 for embedding models, measured over 31 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1E5 Base V2Summarization· MTEB Leaderboard data31.9%
2Granite Embedding 278m MultilingualSummarization· MTEB Leaderboard data31.8%
3Multilingual E5 Large InstructSummarization· MTEB Leaderboard data31.7%
4Qwen3 Embedding 4BSummarization· MTEB Leaderboard data31.3%
5BGE multilingual gemma2Summarization· MTEB Leaderboard data31.3%
6NV-Embed-v2Summarization· MTEB Leaderboard data31.1%
7nomic-embed-text-v1Summarization· MTEB Leaderboard data31.1%
8E5 Mistral 7B InstructSummarization· MTEB Leaderboard data31.1%
9Multilingual E5 LargeSummarization· MTEB Leaderboard data30.9%
10Jina Embeddings v5 Omni NanoSummarization· MTEB Leaderboard data30.7%
11BGE Base En V1.5Summarization· MTEB Leaderboard data30.7%
12Jina Embeddings v3Summarization· MTEB Leaderboard data30.7%
13snowflake-arctic-embed-l-v2.0Summarization· MTEB Leaderboard data30.7%
14Text Embedding Ada 002Summarization· MTEB Leaderboard data30.5%
15Qwen3 Embedding 8BSummarization· MTEB Leaderboard data30.4%
16E5 Large V2Summarization· MTEB Leaderboard data30.4%
17nomic-embed-text-v1.5Summarization· MTEB Leaderboard data30.3%
18Jina Embedding B EN v1Summarization· MTEB Leaderboard data30.2%
19UAE-Large-V1Summarization· MTEB Leaderboard data29.9%
20bge-small-en-v1.5Summarization· MTEB Leaderboard data29.6%
21BGE Large En V1.5Summarization· MTEB Leaderboard data29.5%
22Paraphrase Multilingual MPNet v2Summarization· MTEB Leaderboard data29.5%
23Gte BaseSummarization· MTEB Leaderboard data29.4%
24Gte LargeSummarization· MTEB Leaderboard data29.1%
25Qwen3 Embedding 0.6BSummarization· MTEB Leaderboard data29.1%
26Jina Embeddings v5 Text SmallSummarization· MTEB Leaderboard data29%
27Jina Embeddings v5 Omni SmallSummarization· MTEB Leaderboard data28.8%
28All MiniLM L6 V2Summarization· MTEB Leaderboard data28.3%
29All Mpnet Base V2Summarization· MTEB Leaderboard data28.1%
30Jina Embeddings v2 Base ENSummarization· MTEB Leaderboard data27.7%
31All MiniLM L12 V2Summarization· MTEB Leaderboard data26.6%