Skip to content

Benchmark

FinBen EDTSUM BartScore

Higher scores are better.

Models measured
7
Scores recorded
7
Sources
1
IQ weight
0.65

Score distribution

-3.9
Mean
-3.8
Median
0.268
Std dev
-4
10th percentile
-3.6
90th percentile
-4.5
Min
-4.5-3.6
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.65 for chat models, measured over 7 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1GPT-4 Turbopublished-demo-leaderboard, EDTSUM, BartScore· TheFinAI public numeric results-3.6
2GPT-3.5 Turbopublished-demo-leaderboard, EDTSUM, BartScore· TheFinAI public numeric results-3.6
3DeepSeek V3published-demo-leaderboard, EDTSUM, BartScore· TheFinAI public numeric results-3.8
4Llama 2 70bpublished-demo-leaderboard, EDTSUM, BartScore· TheFinAI public numeric results-3.8
5gemini-1.0-propublished-demo-leaderboard, EDTSUM, BartScore· TheFinAI public numeric results-3.9
6Llama-2-7b-chat-hfpublished-demo-leaderboard, EDTSUM, BartScore· TheFinAI public numeric results-4
7Mixtral 8X7b V0.1published-demo-leaderboard, EDTSUM, BartScore· TheFinAI public numeric results-4.5

Data credits: The Fin AI. Published FinBen result summaries and Open Greek Financial LLM Leaderboard aggregates. Results licence not separately stated; fractional FinBen success metrics converted to percentages.