Skip to content

Benchmark

FinBen ccfraud MCC

Higher scores are better.

Models measured
14
Scores recorded
14
Sources
1
IQ weight
0.66

Score distribution

0.063
Mean
0.007
Median
0.141
Std dev
-0.1
10th percentile
0.283
90th percentile
-0.16
Min
-0.160.335
The band covers the middle 80% of scores. The line marks the median.

This benchmark has a weight of 0.66 for chat models, measured over 14 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.

Leaderboard

#ModelScore
1Qwen2.5 7B Instructpublished-demo-leaderboard, ccfraud, MCC· TheFinAI public numeric results0.335
2codellama-7bpublished-demo-leaderboard, ccfraud, MCC· TheFinAI public numeric results0.283
3Qwen2.5 7Bpublished-demo-leaderboard, ccfraud, MCC· TheFinAI public numeric results0.268
4Mixtral 8X7b V0.1published-demo-leaderboard, ccfraud, MCC· TheFinAI public numeric results0.16
5DeepSeek V3published-demo-leaderboard, ccfraud, MCC· TheFinAI public numeric results0.0924
6Qwen2 7B Instructpublished-demo-leaderboard, ccfraud, MCC· TheFinAI public numeric results0.02
7Mistral (7B) Instruct v0.3published-demo-leaderboard, ccfraud, MCC· TheFinAI public numeric results0.0066
8DeepSeek R1 Distill Qwen 7Bpublished-demo-leaderboard, ccfraud, MCC· TheFinAI public numeric results0
9gemini-1.0-propublished-demo-leaderboard, ccfraud, MCC· TheFinAI public numeric results0
10Llama 2 70bpublished-demo-leaderboard, ccfraud, MCC· TheFinAI public numeric results0
11Llama 3.2 3Bpublished-demo-leaderboard, ccfraud, MCC· TheFinAI public numeric results0
12GPT-4 Turbopublished-demo-leaderboard, ccfraud, MCC· TheFinAI public numeric results-0.02
13GPT-3.5 Turbopublished-demo-leaderboard, ccfraud, MCC· TheFinAI public numeric results-0.1
14Llama-2-7b-chat-hfpublished-demo-leaderboard, ccfraud, MCC· TheFinAI public numeric results-0.16

Data credits: The Fin AI. Published FinBen result summaries and Open Greek Financial LLM Leaderboard aggregates. Results licence not separately stated; fractional FinBen success metrics converted to percentages.