Part of the benchmark canon: 8 entries, all measure the model. How to read the Paladin column is defined on the canon page.
| Benchmark | Organization | Measures | Paladin |
|---|---|---|---|
| CTI-REALM | Microsoft | End-to-end detection-rule generation by agents | Measures the model |
| ExCyTIn-Bench | Microsoft | Agent-driven cyber threat investigation | Measures the model |
| CyberSOCEval | Meta with CrowdStrike | Malware analysis and threat-intel reasoning | Measures the model |
| CTIBench | Academic (Alam et al.) | Threat-intelligence knowledge, attribution, and severity | Measures the model |
| CyberMetric | TII / Khalifa University | Security knowledge question answering | Measures the model |
| SecBench | Tencent / HK PolyU | Large multi-domain security question set | Measures the model |
| SecEval | Academic (Li et al.) | Software, network, and web security questions | Measures the model |
| SecQA | Academic | Foundational security concepts | Measures the model |
Previous: Cyber offense and CTF. Next: Software security and code.