Measured against InjecAgent, a published third-party benchmark, using the benchmark's own scoring code over its full case set.
We publish the caveats next to the numbers. A vendor result you cannot pick apart is not evidence.
InjecAgent is an indirect prompt-injection benchmark from the UIUC Kang Lab (arXiv:2403.02691, MIT licensed). It measures attack success rate: how often a poisoned tool response makes an agent do what the attacker wanted.
We ran the full published case set. All 1,054 cases, 510 Direct Harm and 544 Data Stealing, base setting, no sampling.
The scoring is theirs, not ours. We import InjecAgent's own prompts, their ReAct output parser, and their attack-success formula from their repository, unmodified. The unprotected numbers are produced by the same code path that generates the numbers in the InjecAgent paper. The protected numbers show what changes when Paladin sits in front of the resulting tool call, exactly as a deployment would.
Attack success rate, unprotected against protected, on valid completions.
| Attack class | Model | Unprotected | With Paladin |
|---|---|---|---|
| Data Stealing | Qwen 2.5 7B | 38.8% | 0.0% |
| Data Stealing | Llama 3.2 3B | 1.8% | 0.0% |
| Direct Harm | Qwen 2.5 7B | 28.0% | 15.3% |
Data Stealing was eliminated on both models. Direct Harm was roughly halved on the model that engages meaningfully with the attack.
Roughly half of Direct Harm attacks still succeed. In an exact replay of the 99 Direct Harm cases that succeeded in the published run, holding the model outputs fixed so the result isolates Paladin rather than model drift, Paladin blocked 50 and 49 still got through. An earlier version blocked 45, so three added data classes closed five more cases with no regressions. That is a real improvement and a modest one. It does not close the gap.
The false-positive rate is 22%. Four of 18 benign trials were held. That is a small sample and it is the number we would want to be lower. The measurement followed a fix that roughly halved the previous rate of 5 in 12. The residual source is understood: benign page content occasionally scores above the injection threshold. It is not yet fixed.
The models are small and local. Llama 3.2 3B and Qwen 2.5 7B, run locally through Ollama. Not GPT-class or Claude-class models. If you are running frontier models in production, this result does not yet extend to them, and re-running against those models is the next measurement queued.
Llama's low unprotected rate is mostly a model artifact. It failed to produce a well-formed completion in 48% of cases, for benign and malicious attempts alike. Read its near-zero attack rate as "this model often cannot complete the task format", not as "this model resists injection".
Stage 2 was not run. InjecAgent's second stage asks whether the model completes the exfiltration after a simulated follow-up. Running it needs a paid API key that this run did not use. So Data Stealing and Total figures here reflect Stage 1 only, per InjecAgent's own single-step convention, and are not directly comparable to a figure computed with Stage 2 included.
The enhanced setting was not run. InjecAgent's enhanced setting adds an explicit hacking prompt and would likely show a higher unprotected rate.
Browser agents are measured only directionally so far. A browser-surface harness is committed in the repo: real local models deciding what to do against attack cases sampled from WASP, InjecAgent, and deepset. Across 345 attack trials on three small local models, the layered defense cut successful compromise from unprotected baselines of 10 to 57 percent down to at most 4.3 percent on any one model, and to zero on two of the three. Per-condition samples are small, so read these percentages as directional rather than definitive. They move to the results table above, with dated artifacts, once the run is repeated at full rigor.
An allow adds nothing to the agent's context: zero tokens, no prompt, no second model call. On the proxy path the request and response bodies pass through unchanged and the decision travels in a response header the model never sees.
What an allow does cost is the pass over the tool's output that tells a rule what the payload contains. That cost scales with the size of the output: a short shell result costs almost nothing, and a whole web page costs the most. A measured allow-path figure, with payload sizes and the machine attached, is queued for publication with its artifact. Earlier versions of this page quoted a sub-millisecond p99 for this path. That figure came from a test that exercises the deny path and never calls upstream, so it is retired here until the allow-path measurement is published.
An escalate decision is different by design: it holds the action until a human answers, up to the approval timeout. If you deploy with escalating rules and nobody is watching the console, those actions wait for the timeout and then fail closed. Set the timeout to something your requests can survive.
Each figure comes from a dated artifact, so a stale number is visibly stale.
benchmark-2026-08-02-injecagent-full.jsonbenchmark-2026-08-04-injecagent-dh-detect-replay.jsonbenchmark-2026-08-02-fpr-fix.jsonResults, method, and caveats are public, and every figure in the results table names a dated artifact. Third-party benchmarks are scored with their own code, unmodified, so anyone can rerun the unprotected baseline.
Two things stay off the site on purpose. The exact detection rules and thresholds we run are the mechanism, and we keep them out of the open. And any test corpus we build ourselves stays private, because a published benchmark is training data within months, and a benchmark a model has trained on measures nothing. We publish what it scored, not the questions.
The policy files used in a run are available to teams evaluating Paladin, on request.
Frontier labs and the AI safety institutes cite a canon of 61 AI security and safety benchmarks across 11 categories. One of the 61 has published Paladin results today: the one above. One is queued next. Most of the rest measure the model, not a control plane.
We list all 61, with an honest status on each, so you can see which recognized measurements apply to this product category at all.
How Paladin works, the policy language, and what the default baseline covers: Paladin docs.
A benchmark on someone else's corpus is a starting point, not proof about your agents. If you are evaluating Paladin seriously, the useful exercise is measuring it against your own traffic. That is what we set up with design partners.
Cortexa Labs, Inc.