How it works
The application transforms or tags untrusted text before the model sees it and tells the model never to follow instructions inside it.
Reduces success against fixed attacks but has failed against adaptive attackers.
What we know
1 reported, 1 revalidateRecords over time
Use the arrow keys to move between records, Home and End to jump to the first and last, and Enter to select one.
Select a mark to read the record. Mark size shows editorial significance. Hollow marks are dated to the month. Era bands are editorial labels.
Records in view
2 records · newest firstNasr, Carlini, Tramèr and 11 co-authors apply gradient, reinforcement learning, search and human red-teaming attacks to 12 published defenses. Most defenses originally reported near-zero attack success, but the adaptive attacks exceed 90% success against most, and human red-teamers succeeded on every challenge in the subset of defenses they were given.
Shi and colleagues describe Google DeepMind's continuous adaptive-attack evaluation of Gemini against indirect prompt injection in tool-use settings. On Gemini 2.0, adaptive attacks generally matched or beat non-adaptive ones against eight baseline defenses, reaching 98.4% against in-context learning and 82.4% against spotlighting, while a warning defense and a user-instruction classifier held (at most 10.8% and 3.0%). Adversarial fine-tuning for Gemini 2.5 lowered but did not eliminate attack success.