Measured against InjecAgent, a published third-party benchmark, using the benchmark's own scoring code over its full case set.
We publish the caveats next to the numbers. A vendor result you cannot pick apart is not evidence.
InjecAgent is an indirect prompt-injection benchmark from the UIUC Kang Lab (arXiv:2403.02691, MIT licensed). It measures attack success rate: how often a poisoned tool response makes an agent do what the attacker wanted.
We ran the full published case set. All 1,054 cases, 510 Direct Harm and 544 Data Stealing, base setting, no sampling.
The scoring is theirs, not ours. We import InjecAgent's own prompts, their ReAct output parser, and their attack-success formula from their repository, unmodified. The unprotected numbers are produced by the same code path that generates the numbers in the InjecAgent paper. The protected numbers show what changes when Paladin sits in front of the resulting tool call, exactly as a deployment would.
Attack success rate, unprotected against protected, on valid completions.
| Attack class | Model | Unprotected | With Paladin |
|---|---|---|---|
| Data Stealing | Qwen 2.5 7B | 38.8% | 0.0% |
| Data Stealing | Llama 3.2 3B | 1.8% | 0.0% |
| Direct Harm | Qwen 2.5 7B | 28.0% | 15.3% |
Data Stealing was eliminated on both models. Direct Harm was roughly halved on the model that engages meaningfully with the attack.
Roughly half of Direct Harm attacks still succeed. In an exact replay of the 99 Direct Harm cases that succeeded in the published run, holding the model outputs fixed so the result isolates Paladin rather than model drift, Paladin blocked 50 and 49 still got through. An earlier version blocked 45, so three added data classes closed five more cases with no regressions. That is a real improvement and a modest one. It does not close the gap.
The false-positive rate is 22%. Four of 18 benign trials were held. That is a small sample and it is the number we would want to be lower. The measurement followed a fix that roughly halved the previous rate of 5 in 12. The residual source is understood: benign page content occasionally scores above the injection threshold. It is not yet fixed.
The models are small and local. Llama 3.2 3B and Qwen 2.5 7B, run locally through Ollama. Not GPT-class or Claude-class models. If you are running frontier models in production, this result does not yet extend to them, and re-running against those models is the next measurement queued.
Llama's low unprotected rate is mostly a model artifact. It failed to produce a well-formed completion in 48% of cases, for benign and malicious attempts alike. Read its near-zero attack rate as "this model often cannot complete the task format", not as "this model resists injection".
Stage 2 was not run. InjecAgent's second stage asks whether the model completes the exfiltration after a simulated follow-up. Running it needs a paid API key that this run did not use. So Data Stealing and Total figures here reflect Stage 1 only, per InjecAgent's own single-step convention, and are not directly comparable to a figure computed with Stage 2 included.
The enhanced setting was not run. InjecAgent's enhanced setting adds an explicit hacking prompt and would likely show a higher unprotected rate.
Added latency on the allow path is well under a millisecond at the 99th percentile, inside the 5 ms budget the test suite gates on.
That is the allow path only. An escalate decision is different by design: it holds the action until a human answers, up to the approval timeout. If you deploy with escalating rules and nobody is watching the console, those actions wait for the timeout and then fail closed. Set the timeout to something your requests can survive.
Each figure comes from a dated artifact, so a stale number is visibly stale.
benchmark-2026-08-02-injecagent-full.jsonbenchmark-2026-08-04-injecagent-dh-detect-replay.jsonbenchmark-2026-08-02-fpr-fix.jsonHow Paladin works, the policy language, and what the default baseline covers: Paladin docs.
A benchmark on someone else's corpus is a starting point, not proof about your agents. If you are evaluating Paladin seriously, the useful exercise is measuring it against your own traffic. That is what we set up with design partners.
Cortexa Labs, Inc.