Methods/Defense

Separating control flow from untrusted data

Designing the agent so that untrusted data can never change which actions it takes, only fill in values the plan already allows.

3 records3 defense2 findings (1 measured)First recorded 2024-06assistant-drafted

How it works

A trusted component plans actions from the user's request alone; untrusted content is handled separately and constrained by capabilities, as in CaMeL, the dual-LLM pattern, and plan-then-execute designs.

Known limits

Costs some capability and does not stop manipulation within the actions the plan permits.

What we know

1 corroborated, 1 revalidate

Records over time

RangeLanes
2 of 3 records in view

Use the arrow keys to move between records, Home and End to jump to the first and last, and Enter to select one.

Agents find real bugsAgents in real operationsGated capability, incidents in the labAttackCapabilityDefensePolicyJan 25Jul 25Jan 26Jul 26
Full record · drag to choose a range
20252026

Select a mark to read the record. Mark size shows editorial significance. Hollow marks are dated to the month. Era bands are editorial labels.

Records in view

2 records · newest first
Jun 2025
Jun 10, 2025
Researchers from labs and industry publish design patterns for provably injection-resistant agents
DefenseFrameworkInvariant Labs, IBM, ETH Zurich

Authors from Invariant Labs, IBM, ETH Zurich, Swisscom, Google, Microsoft and others propose six design patterns that limit what injected text can do, such as action-selector, plan-then-execute, dual LLM, LLM map-reduce, code-then-execute and context minimization. They discuss utility and security trade-offs and illustrate the patterns with ten application case studies.

Mar 2025
Mar 24, 2025
Google DeepMind's CaMeL defeats prompt injections by design with capability-based control and data flow
DefensePaperGoogle DeepMind, Google, ETH Zurich

Debenedetti and colleagues (Google, Google DeepMind, ETH Zurich) propose CaMeL, which extracts control flow from the trusted user query so untrusted data cannot change which actions run, and attaches capabilities to data to block unauthorized flows. On AgentDojo the first version reported 67% of tasks solved with provable security; the June 2025 revision, with newer models, reports 77% versus 84% for an undefended system.

All records