Part of the benchmark canon: 4 entries, all measure the model. How to read the Paladin column is defined on the canon page.
| Benchmark | Organization | Measures | Paladin |
|---|---|---|---|
| METR RE-Bench (top 12) | METR | AI R&D against human expert baselines | Measures the model |
| METR HCAST | METR | Task length a model can complete, tracked over time | Measures the model |
| MLAgentBench | Stanford | Autonomous machine-learning research tasks | Measures the model |
| SWE-bench Verified | OpenAI Preparedness / Princeton NLP | Human-verified real GitHub issues | Measures the model |
Previous: Alignment, honesty, scheming. Next: Comprehensive safety and trust.