Agentic Cyber Explorerby Fide AI · PreviewPreview
Chronicle/Defense & research

Attention-based injection detection transfers poorly to an AgentDojo subset in a cross-dataset study

DefensePaperSignificance assistant-drafted

Amelia Kawasaki and Stefan Lee evaluate Attention Tracker and propose AttnDA, a learned alternative using attention-head features. Across five tested open models, synthetic-trained Attention Tracker averages 0.97 AUROC on deepset but 0.57 on the paper’s AgentDojo subset. Domain-matched training helps AttnDA; these are detection results rather than evidence of preventing harmful agent actions.

Why it matters

It directly tests whether a detector’s single-turn benchmark performance transfers to structured agent tool outputs.

Key facts

As stated in the sources, with where to find them.

  • The cited arXiv version 1 is dated 2026-10-20. The AgentDojo subset uses the banking and Slack suites, with 25 and 26 samples respectively; results do not cover the full AgentDojo benchmark.Title page; Section 5, Datasets
  • Table 1 reports average AUROC of 0.57 for Attention Tracker and 0.62 for AttnDA on this AgentDojo subset when trained on synthetic data, versus 0.97 and 0.95 on deepset. AUROC is not accuracy or attack-prevention rate.Table 1; Section 5, Metrics
  • Tests cover Llama-3-8B-Instruct, Qwen2-1.5B, Gemma 2 9B, Phi-3 mini and Mistral 7B. They use attention access, evaluate injected tool-result detection, and omit a system prompt following the original method’s procedure.Section 5, Agentic Evaluation Procedure; Models
  • The paper notes small-sample uncertainty and poor transfer into AgentDojo. Both attention methods materialize attention maps: at 2,048 tokens the reported memory footprint rises from 1.7 GB baseline to 9.6 GB, despite AttnDA adding no measured cost relative to Attention Tracker.Sections 6.8 and 7.2

Findings that cite this record

No tracked finding cites this record yet.

Sources

Related records