StruQ

Nasr, Carlini, Tramèr and 11 co-authors apply gradient, reinforcement learning, search and human red-teaming attacks to 12 published defenses. Most defenses originally reported near-zero attack success, but the adaptive attacks exceed 90% success against most, and human red-teamers succeeded on every challenge in the subset of defenses they were given.

Chen, Piet, Sitawarin and Wagner propose structured queries, in which a secure front-end separates the trusted prompt from untrusted data and a model fine-tuned to ignore instructions appearing in the data portion. The paper reports much better resistance to prompt injection with little utility loss; it appeared at USENIX Security 2025.