Skip to content

Benchmark

CodeMemoryBench · QA score

Scores are percentages. Higher is better.

Models and systems measured
2
Scores recorded
14
Sources
1
IQ weight
No weight published

Score distribution

45.5%
Mean
51.4%
Median
15.1%
Std dev
18.7%
10th percentile
62.2%
90th percentile
9.6%
Min
9.6%64.2%
The band covers the middle 80% of scores. The line marks the median.

Results

The source marks these results as descriptive, not capability scores, so we list them by name instead of ranking them.

Model or systemScore
Claude Sonnet 5 (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: FullContext · candidate snapshot, reasoning effort, and run count not stated · 750 QA questions; judge-derived requirement-match mean· Reported by Tencent Hunyuan· via BenchBoard docs56.1%
Claude Sonnet 5 (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: Mem0 · candidate snapshot, reasoning effort, and run count not stated · 750 QA questions; judge-derived requirement-match mean· Reported by Tencent Hunyuan· via BenchBoard docs37.8%
Claude Sonnet 5 (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: MemOS · candidate snapshot, reasoning effort, and run count not stated · 750 QA questions; judge-derived requirement-match mean· Reported by Tencent Hunyuan· via BenchBoard docs51.9%
Claude Sonnet 5 (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: OpenViking · candidate snapshot, reasoning effort, and run count not stated · 750 QA questions; judge-derived requirement-match mean· Reported by Tencent Hunyuan· via BenchBoard docs46.8%
Claude Sonnet 5 (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: RAG · candidate snapshot, reasoning effort, and run count not stated · 750 QA questions; judge-derived requirement-match mean· Reported by Tencent Hunyuan· via BenchBoard docs48%
Claude Sonnet 5 (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: RepoMemory · candidate snapshot, reasoning effort, and run count not stated · 750 QA questions; judge-derived requirement-match mean· Reported by Tencent Hunyuan· via BenchBoard docs18.7%
Claude Sonnet 5 (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: TriLink · candidate snapshot, reasoning effort, and run count not stated · 750 QA questions; judge-derived requirement-match mean· Reported by Tencent Hunyuan· via BenchBoard docs58.2%
DeepSeek V4 Flash (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: FullContext · candidate snapshot, reasoning effort, and run count not stated · 750 QA questions; judge-derived requirement-match mean· Reported by Tencent Hunyuan· via BenchBoard docs53.4%
DeepSeek V4 Flash (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: Mem0 · candidate snapshot, reasoning effort, and run count not stated · 750 QA questions; judge-derived requirement-match mean· Reported by Tencent Hunyuan· via BenchBoard docs40%
DeepSeek V4 Flash (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: MemOS · candidate snapshot, reasoning effort, and run count not stated · 750 QA questions; judge-derived requirement-match mean· Reported by Tencent Hunyuan· via BenchBoard docs51.4%
DeepSeek V4 Flash (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: OpenViking · candidate snapshot, reasoning effort, and run count not stated · 750 QA questions; judge-derived requirement-match mean· Reported by Tencent Hunyuan· via BenchBoard docs38.6%
DeepSeek V4 Flash (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: RAG · candidate snapshot, reasoning effort, and run count not stated · 750 QA questions; judge-derived requirement-match mean· Reported by Tencent Hunyuan· via BenchBoard docs62.2%
DeepSeek V4 Flash (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: RepoMemory · candidate snapshot, reasoning effort, and run count not stated · 750 QA questions; judge-derived requirement-match mean· Reported by Tencent Hunyuan· via BenchBoard docs9.6%
DeepSeek V4 Flash (CMB paper label)CodeMemoryBench paper run · mini-swe-agent · frozen histories · memory: TriLink · candidate snapshot, reasoning effort, and run count not stated · 750 QA questions; judge-derived requirement-match mean· Reported by Tencent Hunyuan· via BenchBoard docs64.2%

Data credits: BenchBoard by SYuan03 (github.com/SYuan03/benchboard). MIT covers its code and data organization; no results licence is inferred. Numeric relays retain the original stating-source links.