2024-06-19
Undefended tool-using agents follow injected instructions in a substantial share of benchmark cases.
AgentDojo, from a different group, measures the same failure.
Week of Jun 17–23, 2024
1 records1 status changes on new evidence0 new findings
What changed in what we know
Defense & research
Jun 19, 2024
AgentDojo: an extensible environment for prompt injection attacks and defenses on LLM agents
Debenedetti and colleagues (ETH Zurich, Invariant Labs) release AgentDojo, a dynamic environment with 97 realistic user tasks across workspace, banking, travel and Slack suites and 629 security test cases. It measures both utility and targeted attack success, and reports that existing attacks break some security properties but not all. It became the standard testbed used by CaMeL, US AISI/CAISI, LlamaFirewall and adaptive-attack studies.