Benchmark
Physical AI Bench Understanding
Vision benchmarks measure how well models understand images. Scores are percentages.
- Models measured
- 15
- Scores recorded
- 17
- Sources
- 1
- IQ weight
- 1.21
Score distribution
- 59%
- Mean
- 59.2%
- Median
- 5.5%
- Std dev
- 55.3%
- 10th percentile
- 64.7%
- 90th percentile
- 46%
- Min
This benchmark has a weight of 1.21 for chat models, measured over 15 models. The weight is how much it counts toward llminfo IQ. We measure it from the data, not from the raw scores.
Leaderboard
| # | Model | Score |
|---|---|---|
| 1 | GPT 5thinking· Physical AI Bench data | 69.8% |
| 2 | Cosmos Reason2 8b· Physical AI Bench data | 65.7% |
| 3 | Qwen3 VL 235B· Physical AI Bench data | 64.7% |
| 4 | Qwen3 VL 235Bthinking· Physical AI Bench data | 63.7% |
| 5 | Qwen3 VL 32B Instruct· Physical AI Bench data | 62% |
| 6 | Qwen3 VL 32B Thinkingthinking· Physical AI Bench data | 61% |
| 7 | Qwen2.5 VL 72B Instruct· Physical AI Bench data | 60.8% |
| 8 | qwen3-vl-30b· Physical AI Bench data | 59.5% |
| 9 | GLM 4.5V· Physical AI Bench data | 59.2% |
| 10 | Qwen: Qwen3 VL 8B Thinkingthinking· Physical AI Bench data | 57.3% |
| 11 | qwen3-vl-30bthinking· Physical AI Bench data | 57.3% |
| 12 | Qwen3 VL 8B Instruct· Physical AI Bench data | 56.8% |
| 13 | internvl3-5-241b-a28b· Physical AI Bench data | 56.3% |
| 14 | GPT 4o· Physical AI Bench data | 56.2% |
| 15 | Qwen 2.5 VL 32B Instruct· Physical AI Bench data | 55.3% |
| 16 | Qwen2.5-VL 7B Instruct· Physical AI Bench data | 51% |
| 17 | Claude 3.5 Sonnet· Physical AI Bench data | 46% |
Data credits: Physical AI Bench (SHI Labs), MIT (licence)