Skip to content

Benchmark

FinBen MultiFin-ES Macro F1

Scores are percentages. Higher is better.

Models measured
6
Scores recorded
6
Sources
1
IQ weight
1.56

Score distribution

34.5%
Mean
39%
Median
12.1%
Std dev
32.7%
10th percentile
49.3%
90th percentile
10.1%
Min
10.1%49.3%
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 1.56 for chat models, measured over 6 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Qwen2.5 7B Instructpublished-demo-leaderboard, MultiFin-ES, Macro F1· TheFinAI public numeric results49.3%
2Llama 3.2 3Bpublished-demo-leaderboard, MultiFin-ES, Macro F1· TheFinAI public numeric results39.7%
3codellama-7bpublished-demo-leaderboard, MultiFin-ES, Macro F1· TheFinAI public numeric results39.1%
4Qwen2.5 7Bpublished-demo-leaderboard, MultiFin-ES, Macro F1· TheFinAI public numeric results36.4%
5Mistral (7B) Instruct v0.3published-demo-leaderboard, MultiFin-ES, Macro F1· TheFinAI public numeric results32.7%
6code-llama-7b-instructpublished-demo-leaderboard, MultiFin-ES, Macro F1· TheFinAI public numeric results10.1%

Data credits: The Fin AI. Published FinBen result summaries and Open Greek Financial LLM Leaderboard aggregates. Results licence not separately stated; fractional FinBen success metrics converted to percentages.