Skip to content

Benchmark

FollowIR

Embedding benchmarks measure how well models find related items, for example when matching a search to relevant results. Scores are percentages.

Models measured
40
Scores recorded
40
Sources
1
IQ weight
0.85

Score distribution

-0.429%
Mean
-1.3%
Median
3.5%
Std dev
-3.3%
10th percentile
4.1%
90th percentile
-4.3%
Min
-4.3%11.6%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.85 for embedding models, measured over 40 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Qwen3 Embedding 4B· MTEB Extra Text Boards data11.6%
2Qwen3 Embedding 8B· MTEB Extra Text Boards data10.1%
3Gemini Embedding 001· MTEB Extra Text Boards data5.2%
4Qwen3 Embedding 0.6B· MTEB Extra Text Boards data5.1%
5Text Multilingual Embedding 002· MTEB Extra Text Boards data4.1%
6INF Retriever v1· MTEB Extra Text Boards data2.6%
7Text Embedding 004· MTEB Extra Text Boards data2.3%
8Jina Embeddings v5 Omni Small· MTEB Extra Text Boards data1.4%
9Jina Embeddings v5 Text Small· MTEB Extra Text Boards data1.4%
10NV-Embed-v2· MTEB Extra Text Boards data1%
11Jina Embeddings v5 Omni Nano· MTEB Extra Text Boards data0.05%
12Jina Embeddings v5 Text Nano· MTEB Extra Text Boards data0.05%
13Multilingual E5 Large Instruct· MTEB Extra Text Boards data-0.4%
14embeddinggemma-2· MTEB Extra Text Boards data-0.61%
15E5 Mistral 7B Instruct· MTEB Extra Text Boards data-0.62%
16All MiniLM L12 V2· MTEB Extra Text Boards data-0.75%
17E5 Large V2· MTEB Extra Text Boards data-0.99%
18Paraphrase Multilingual MPNet v2· MTEB Extra Text Boards data-1.1%
19Gte Large· MTEB Extra Text Boards data-1.3%
20Jina Embeddings v3· MTEB Extra Text Boards data-1.3%
21Jina Embeddings v2 Base EN· MTEB Extra Text Boards data-1.6%
22Granite Embedding 278m Multilingual· MTEB Extra Text Boards data-1.8%
23nomic-embed-text-v1.5· MTEB Extra Text Boards data-1.9%
24bge-small-en-v1.5· MTEB Extra Text Boards data-1.9%
25UAE-Large-V1· MTEB Extra Text Boards data-2.1%
26BGE Large En V1.5· MTEB Extra Text Boards data-2.3%
27bge-large-en· MTEB Extra Text Boards data-2.4%
28Text Embedding 3 Small· MTEB Extra Text Boards data-2.4%
29Gte Base· MTEB Extra Text Boards data-2.5%
30snowflake-arctic-embed-l-v2.0· MTEB Extra Text Boards data-2.5%
31Text Embedding 3 Large· MTEB Extra Text Boards data-2.7%
32All MiniLM L6 V2· MTEB Extra Text Boards data-2.8%
33Multilingual E5 Large· MTEB Extra Text Boards data-3.1%
34All Mpnet Base V2· MTEB Extra Text Boards data-3.1%
35BGE M3· MTEB Extra Text Boards data-3.1%
36nomic-embed-text-v1· MTEB Extra Text Boards data-3.3%
37snowflake-arctic-embed-m-v2.0· MTEB Extra Text Boards data-3.3%
38E5 Base V2· MTEB Extra Text Boards data-3.9%
39BGE Base En V1.5· MTEB Extra Text Boards data-4%
40Jina Embedding B EN v1· MTEB Extra Text Boards data-4.3%