Shi and colleagues describe Google DeepMind's continuous adaptive-attack evaluation of Gemini against indirect prompt injection in tool-use settings. On Gemini 2.0, adaptive attacks generally matched or beat non-adaptive ones against eight baseline defenses, reaching 98.4% against in-context learning and 82.4% against spotlighting, while a warning defense and a user-instruction classifier held (at most 10.8% and 3.0%). Adversarial fine-tuning for Gemini 2.5 lowered but did not eliminate attack success.
Why it matters
A frontier developer documented that static-benchmark defense numbers overstate robustness.
Key facts
As stated in the sources, with where to find them.
- Against undefended Gemini 2.0, at least one of three adaptive attacks succeeded on over 70% of test examples in every setting; TAP reached close to 100% in nearly all settings.Section 6, Figure 4
- Adaptive attacks on Gemini 2.0 with defenses (calendar, passport scenario): peak ASR 98.4% against ICL, 95.2% against paraphrasing and 82.4% against spotlighting; Warning at most 10.8% and the user-instruction classifier at most 3.0%.Section 8; Appendix E, Table 4
- In 16 of 24 defense-attack pairs the adaptive attack matched or beat the non-adaptive one.Section 8.2, Figure 6
- Adversarially trained Gemini 2.5 vs Gemini 2.0, no external defenses: email/passport TAP 53.6% vs 99.8% and Actor-Critic 40.8% vs 66.2%; calendar/passport TAP 94.6% vs 100%. Adding the Warning defense to Gemini 2.5 cut calendar TAP to 6.2%.Section 9.1, Table 1; Section 9.2
Findings that cite this record
Key questions this bears on
- Can prompt injection against AI agents be reliably defended?Not reliably. Bounding what untrusted input can trigger helps, but trust boundaries must survive delegation and tool execution.
Sources
Related records
Oct 10, 2025
Mar 24, 2025
Sep 29, 2026
Jun 11, 2025
Nov 9, 2026
Oct 16, 2026