Skip to content

Benchmark

FinBen ACL18 MCC

Higher scores are better.

Models measured
15
Scores recorded
15
Sources
1
IQ weight
0.00

Score distribution

0.013
Mean
0.007
Median
0.021
Std dev
-0.014
10th percentile
0.044
90th percentile
-0.02
Min
-0.020.0603
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.00 for chat models, measured over 15 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1DeepSeek V3published-demo-leaderboard, ACL18, MCC· TheFinAI public numeric results0.0603
2DeepSeek R1 Distill Qwen 1.5Bpublished-demo-leaderboard, ACL18, MCC· TheFinAI public numeric results0.0444
3gemini-1.0-propublished-demo-leaderboard, ACL18, MCC· TheFinAI public numeric results0.04
4Mistral (7B) Instruct v0.3published-demo-leaderboard, ACL18, MCC· TheFinAI public numeric results0.0253
5GPT-4 Turbopublished-demo-leaderboard, ACL18, MCC· TheFinAI public numeric results0.02
6Llama 2 70bpublished-demo-leaderboard, ACL18, MCC· TheFinAI public numeric results0.01
7Llama-2-7b-chat-hfpublished-demo-leaderboard, ACL18, MCC· TheFinAI public numeric results0.01
8Qwen2.5 7B Instructpublished-demo-leaderboard, ACL18, MCC· TheFinAI public numeric results0.0071
9GPT-3.5 Turbopublished-demo-leaderboard, ACL18, MCC· TheFinAI public numeric results0.005
10DeepSeek R1 Distill Qwen 7Bpublished-demo-leaderboard, ACL18, MCC· TheFinAI public numeric results0.0024
11Llama 3.2 3Bpublished-demo-leaderboard, ACL18, MCC· TheFinAI public numeric results0.0015
12codellama-7bpublished-demo-leaderboard, ACL18, MCC· TheFinAI public numeric results0
13Mixtral 8X7b V0.1published-demo-leaderboard, ACL18, MCC· TheFinAI public numeric results0
14Qwen2.5 7Bpublished-demo-leaderboard, ACL18, MCC· TheFinAI public numeric results-0.0136
15Qwen2 7B Instructpublished-demo-leaderboard, ACL18, MCC· TheFinAI public numeric results-0.02

Data credits: The Fin AI. Published FinBen result summaries and Open Greek Financial LLM Leaderboard aggregates. Results licence not separately stated; fractional FinBen success metrics converted to percentages.