{
 "license": "CC-BY-4.0",
 "attribution": "Fide AI, Agentic Cyber Explorer",
 "url": "https://agentic-cyber-explorer.pages.dev/events/ignored-not-refused-prompt-injection-small-agents-2026/",
 "asOf": "2026-10-30",
 "id": "ignored-not-refused-prompt-injection-small-agents-2026",
 "date": "2026-10-16",
 "datePrecision": "day",
 "title": "BRAC University audit finds 76-80% of failed prompt-injection attacks on small open-weight agents were ignored, not refused",
 "lane": "defense",
 "kind": "paper",
 "summary": "BRAC University researchers audit 33,000 trajectories from eleven open-weight agents (2-14B parameters) on InjecAgent and AgentDojo with a label-blind LLM judge, and report that 76.3% (AgentDojo) and 80.3% (InjecAgent) of attacks the benchmarks scored as failed were ignored by the agent, against 18.0% and 17.2% genuinely refused. Holding models and attacks fixed, they report that a system prompt permitting instructions in tool outputs raised pooled InjecAgent attack success from 8.09% at baseline to 43.05%, while a matched prohibiting prompt lowered it to 0.22%. The study uses static injections, greedy decoding and a single LLM judge, and does not test adaptive attackers or live deployments.",
 "whyItMatters": "It shows that a low attack success rate on these two benchmarks can reflect agents that never acted on the injected text, so the rate alone may overstate robustness.",
 "actors": [
  "brac-university"
 ],
 "topics": [
  "prompt-injection",
  "eval-validity"
 ],
 "atlas": [
  "untrusted-content",
  "model",
  "eval-environment"
 ],
 "artifacts": [
  "agentdojo",
  "injecagent",
  "kimi"
 ],
 "sources": [
  {
   "url": "https://arxiv.org/abs/2610.23254",
   "publisher": "arXiv",
   "title": "Ignored, Not Refused: What Low Prompt-Injection Success Rates Hide in Small Tool-Using Agents",
   "date": "2026-10-16",
   "type": "primary",
   "accessed": "2026-10-19"
  }
 ],
 "keyFacts": [
  {
   "fact": "Setup: eleven open-weight agents from six architecture families (2-14B parameters), each served at FP16, FP8 and NF4, scored on InjecAgent (1,054 cases) and AgentDojo (949 episodes). A locked random sample of 500 trajectories per agent, benchmark and precision gives 33,000 audited trajectories. The label-blind judge is DeepSeek-V4.1-Flash, chosen because no evaluated agent is from the DeepSeek family; decoding is greedy.",
   "locator": "v1, Section 3 (Study Design)"
  },
  {
   "fact": "Baseline FP16 pooled attack success rate (ASR) is 8.09% on InjecAgent and 11.96% on AgentDojo; eight of eleven agents are below 10% on InjecAgent. On InjecAgent the parser rejected 1,562 FP16 outputs as invalid; for example LFM-2.6B has 0.00% ASR but 735 of 1,054 outputs invalid, so most of its cases never test resistance.",
   "locator": "v1, Section 4.1; Table 3"
  },
  {
   "fact": "Of attacks scored as failed (14,699 on AgentDojo, 15,274 on InjecAgent), the judge labels 76.3% and 80.3% as ignored (injection never treated as an instruction, or recognized but neither executed nor rejected) and 18.0% and 17.2% as genuine refusals. The paper says the ignored-over-refused pattern holds at every precision and under every robustness check it ran.",
   "locator": "v1, Section 4.1; Table 2"
  },
  {
   "fact": "Recognition is not resistance: the judge flagged recognition of the injection in 4,740 AgentDojo and 5,688 InjecAgent trajectories, and in 28.9% and 16.9% of them the agent went on to execute the attacker's action. Recognition is read from stated reasoning only, which the authors call a lower bound.",
   "locator": "v1, Section 4.1; Limitations"
  },
  {
   "fact": "Frontier reference, one run of one model: Kimi K3 at FP16 had ASR 0.4% on AgentDojo (4 of 949 attacks) and 0% on InjecAgent, and 96.6% and 99.0% of its failed attacks were genuine refusals. The authors state this is a reference point, not a scaling result.",
   "locator": "v1, Section 4.1; Appendix F"
  },
  {
   "fact": "Deployment sensitivity, InjecAgent only: on the 10,698 cases run under both policies, a system prompt permitting instructions in tool outputs gave pooled ASR 43.05% and a matched prompt forbidding them gave 0.22%, against 8.09% at baseline; the two prompts differ by nine words. Nine of eleven agents shifted significantly in paired tests. A three-rule source-boundary guardrail lowered pooled ASR from 9.69% to 0.95% across nine agents. The authors read the low baseline as agents not being triggered, not as an enforced boundary.",
   "locator": "v1, Section 4.2; Figure 2"
  },
  {
   "fact": "Serving precision: FP8 and NF4 lowered pooled ASR only slightly (InjecAgent 8.09% to 7.50% and 6.35%; AgentDojo 11.96% to 11.11% and 9.63%) with refusal rates flat; the authors attribute the drop to weaker agent behavior, such as more invalid outputs, not better defense.",
   "locator": "v1, Section 4.1; Appendix Table 9"
  },
  {
   "fact": "Attack-Task Outcome Matrix on 16,273 audited AgentDojo episodes: 67.2% safe and task completed, 21.7% safe but task not completed, 3.8% attack succeeded with task completed, 7.3% attack succeeded and task not completed. Three agents with near-equal net resilient performance scores (81.0, 80.6, 80.6) differ widely in these cells. Task completion is measured only on AgentDojo.",
   "locator": "v1, Section 4.3; Figure 3; Table 5"
  },
  {
   "fact": "Judge validation: two authors labeled 150 trajectories and agree with the judge on every attacker action and on 96% of four-way groups (kappa 0.835); the judge agrees with benchmark verdicts on 96.7% of AgentDojo and 98.5% of InjecAgent cases. Stated limits: two synthetic single-agent benchmarks with static injections, no adaptive or white-box attackers, one LLM judge, and policy conditions run only on InjecAgent.",
   "locator": "v1, Section 3; Limitations"
  }
 ],
 "significance": 3,
 "fideQuestions": [
  "FID-075"
 ],
 "methods": [
  "indirect-prompt-injection"
 ],
 "review": "assistant-drafted",
 "addedOn": "2026-10-19",
 "revisions": []
}