BRAC University audit finds 76-80% of failed…

BRAC University researchers audit 33,000 trajectories from eleven open-weight agents (2-14B parameters) on InjecAgent and AgentDojo with a label-blind LLM judge, and report that 76.3% (AgentDojo) and 80.3% (InjecAgent) of attacks the benchmarks scored as failed were ignored by the agent, against 18.0% and 17.2% genuinely refused. Holding models and attacks fixed, they report that a system prompt permitting instructions in tool outputs raised pooled InjecAgent attack success from 8.09% at baseline to 43.05%, while a matched prohibiting prompt lowered it to 0.22%. The study uses static injections, greedy decoding and a single LLM judge, and does not test adaptive attackers or live deployments.

Why it matters

It shows that a low attack success rate on these two benchmarks can reflect agents that never acted on the injected text, so the rate alone may overstate robustness.

Key facts

As stated in the sources, with where to find them.

Setup: eleven open-weight agents from six architecture families (2-14B parameters), each served at FP16, FP8 and NF4, scored on InjecAgent (1,054 cases) and AgentDojo (949 episodes). A locked random sample of 500 trajectories per agent, benchmark and precision gives 33,000 audited trajectories. The label-blind judge is DeepSeek-V4.1-Flash, chosen because no evaluated agent is from the DeepSeek family; decoding is greedy.v1, Section 3 (Study Design)
Baseline FP16 pooled attack success rate (ASR) is 8.09% on InjecAgent and 11.96% on AgentDojo; eight of eleven agents are below 10% on InjecAgent. On InjecAgent the parser rejected 1,562 FP16 outputs as invalid; for example LFM-2.6B has 0.00% ASR but 735 of 1,054 outputs invalid, so most of its cases never test resistance.v1, Section 4.1; Table 3
Of attacks scored as failed (14,699 on AgentDojo, 15,274 on InjecAgent), the judge labels 76.3% and 80.3% as ignored (injection never treated as an instruction, or recognized but neither executed nor rejected) and 18.0% and 17.2% as genuine refusals. The paper says the ignored-over-refused pattern holds at every precision and under every robustness check it ran.v1, Section 4.1; Table 2
Recognition is not resistance: the judge flagged recognition of the injection in 4,740 AgentDojo and 5,688 InjecAgent trajectories, and in 28.9% and 16.9% of them the agent went on to execute the attacker's action. Recognition is read from stated reasoning only, which the authors call a lower bound.v1, Section 4.1; Limitations
Frontier reference, one run of one model: Kimi K3 at FP16 had ASR 0.4% on AgentDojo (4 of 949 attacks) and 0% on InjecAgent, and 96.6% and 99.0% of its failed attacks were genuine refusals. The authors state this is a reference point, not a scaling result.v1, Section 4.1; Appendix F
Deployment sensitivity, InjecAgent only: on the 10,698 cases run under both policies, a system prompt permitting instructions in tool outputs gave pooled ASR 43.05% and a matched prompt forbidding them gave 0.22%, against 8.09% at baseline; the two prompts differ by nine words. Nine of eleven agents shifted significantly in paired tests. A three-rule source-boundary guardrail lowered pooled ASR from 9.69% to 0.95% across nine agents. The authors read the low baseline as agents not being triggered, not as an enforced boundary.v1, Section 4.2; Figure 2
Serving precision: FP8 and NF4 lowered pooled ASR only slightly (InjecAgent 8.09% to 7.50% and 6.35%; AgentDojo 11.96% to 11.11% and 9.63%) with refusal rates flat; the authors attribute the drop to weaker agent behavior, such as more invalid outputs, not better defense.v1, Section 4.1; Appendix Table 9
Attack-Task Outcome Matrix on 16,273 audited AgentDojo episodes: 67.2% safe and task completed, 21.7% safe but task not completed, 3.8% attack succeeded with task completed, 7.3% attack succeeded and task not completed. Three agents with near-equal net resilient performance scores (81.0, 80.6, 80.6) differ widely in these cells. Task completion is measured only on AgentDojo.v1, Section 4.3; Figure 3; Table 5
Judge validation: two authors labeled 150 trajectories and agree with the judge on every attacker action and on 96% of four-way groups (kappa 0.835); the judge agrees with benchmark verdicts on 96.7% of AgentDojo and 98.5% of InjecAgent cases. Stated limits: two synthetic single-agent benchmarks with static injections, no adaptive or white-box attackers, one LLM judge, and policy conditions run only on InjecAgent.v1, Section 3; Limitations

Findings that cite this record

No tracked finding cites this record yet.

Key questions this bears on

Can prompt injection against AI agents be reliably defended?Not reliably. Bounding what untrusted input can trigger helps, but trust boundaries must survive delegation and tool execution.