Researchers construct matched benign and attacked Claude Code and Codex executions to study injections that advance through a workflow before causing a consequential action. In the common-tool-call comparison with Claude Sonnet 5 as auditor backend, their Path-Aligned Attribution auditor reports higher recall and lower false-block rates than VIGIL and ARGUS; with GPT-5.5, results involve a recall–false-block tradeoff. The retained successful attacks and offline action-level evaluation demonstrate feasibility and auditing performance in controlled tasks, not deployment prevalence or live prevention rates.
It measures a defense at the point where an action can still be stopped, while distinguishing benign task continuation from unauthorized effects.
Key facts
As stated in the sources, with where to find them.
- The benchmark contains 479 matched attack–benign trajectory pairs and 3,112 pending-action units across eight workflow scenarios, seven attack goals and six injection surfaces. Retention selects confirmed successful staged attacks, not a uniform sample of attempted attacks.Abstract; Introduction
- With Claude Sonnet 5 and full-benchmark fail-open scoring, the proposed auditor reports 86% BLOCK recall and a 6–8% false-block rate, versus ARGUS at 44–47% recall and 16–33% false-block rate. These classify action units; they are not the fraction of whole attacks prevented in deployment. Unsupported action boundaries and auditor failures count as PASS under this fail-open comparison. In the common-tool-call view with the same backend, the authors report higher recall and lower false-block rates than both VIGIL and ARGUS on each corpus.Abstract; Introduction; Evaluation
- Attack schemes are fixed before the target execution and do not adapt during it. Native agent runtimes operate in isolated workspaces with controlled scenario tools; the study does not target real customer accounts or production services.Introduction; Ethical Considerations
- The authors describe boundary auditing as complementary to isolation and permission enforcement. Successful retained cases do not establish prevalence, repeated success of a fixed scheme, unseen-task performance or a security ranking of the two agent products.Discussion and Limitations
Findings that cite this record
No tracked finding cites this record yet.
Key questions this bears on
- Can prompt injection against AI agents be reliably defended?Not reliably. Bounding what untrusted input can trigger helps, but trust boundaries must survive delegation and tool execution.
- Can AI agents defend and oversee systems on their own?Not reliably. Some designs improve benchmark triage, but autonomous defense and oversight lack broad operational validation.