CapLite derives write permissions from tool schemas and values in the user request before reading untrusted tool results. The preprint’s authors report no successful tool-goal attacks in 13,935 AgentDojo episodes across 15 open-weight models, and 29.0% utility under attack on a broader task set. Its conditional write-integrity guarantee excludes output-only goals and does not establish read confidentiality or general agent security.
Prompt injection
Instructions smuggled into an agent through the content it reads.
Prompt injection in tool-using agents
From the first description of indirect prompt injection to why published defenses fail against adaptive attackers, and what design choices hold up.
- Researchers define indirect prompt injection against LLM-integrated apps including Bing ChatWhere the problem was first described.
- AgentDojo: an extensible environment for prompt injection attacks and defenses on LLM agentsHow it is measured in tool-using agents.
- Undefended tool-using agents follow injected instructions in a substantial share of benchmark cases.What undefended agents do.
- EchoLeak: zero-click prompt injection in Microsoft 365 Copilot (CVE-2025-32711)A zero-click case in a shipping product.
- Attackers who adapt to a defense defeat most published prompt-injection defenses that reported near-zero success against static attacks.Why static defense results mislead.
- Limiting what untrusted input can cause an agent to do gives injection resistance that does not depend on the model resisting.The design principle that does not depend on the model.
- Do production prompt-injection defenses survive independent adaptive attack?The open question.
Use the arrow keys to move between records, Home and End to jump to the first and last, and Enter to select one.
Select a mark to read the record. Mark size shows editorial significance. Hollow marks are dated to the month. Era bands are editorial labels.
Records in view
81 records · newest firstA Google DeepMind preprint tests collaborative attacker agents on a synthetic instruction-hierarchy benchmark with randomly selected, non-malicious tool-call goals. Stronger attackers and larger query budgets expose failures that static tests miss. The experiments isolate model behavior and do not measure successful real-world data theft or a complete deployed defense stack.
Forever Security reports embedded-browser weaknesses in Cursor, GitHub Copilot, Claude Desktop, Codex and VS Code, with different impacts including host execution, address spoofing and local-file access. The researchers describe controlled demonstrations and product-specific fixes or unresolved reports; their account does not establish exploitation against users.
Shilov and colleagues introduce Heads of Security, learned attention gates separating trusted instructions from untrusted content, tested on GPT-OSS-20B and Llama-3.3-70B-Instruct. Their preprint reports improved robustness against adaptive attacks while preserving tested utility; the mechanism depends on externally supplied trust labels and is not an end-to-end authorization system.
Amelia Kawasaki and Stefan Lee evaluate Attention Tracker and propose AttnDA, a learned alternative using attention-head features. Across five tested open models, synthetic-trained Attention Tracker averages 0.97 AUROC on deepset but 0.57 on the paper’s AgentDojo subset. Domain-matched training helps AttnDA; these are detection results rather than evidence of preventing harmful agent actions.
Researchers from Fullive-AI, Peking University, JD.com and other institutions replay AgentDojo episodes up to the step where an agent reads an injected tool output, then resample the rest many times to separate what is fixed at read time from what is decided afterwards. They report that a Qwen3-8B agent without thinking is largely decided at read time, while with thinking part of the decision forms during reasoning. Zero-shot chain-of-thought monitors on cheaper models miss much of what is already decided, while stronger monitors and a lookahead monitor that judges simulated next actions come close to the measured ceiling.
BRAC University researchers audit 33,000 trajectories from eleven open-weight agents (2-14B parameters) on InjecAgent and AgentDojo with a label-blind LLM judge, and report that 76.3% (AgentDojo) and 80.3% (InjecAgent) of attacks the benchmarks scored as failed were ignored by the agent, against 18.0% and 17.2% genuinely refused. Holding models and attacks fixed, they report that a system prompt permitting instructions in tool outputs raised pooled InjecAgent attack success from 8.09% at baseline to 43.05%, while a matched prohibiting prompt lowered it to 0.22%. The study uses static injections, greedy decoding and a single LLM judge, and does not test adaptive attackers or live deployments.
Johann Rehberger (Embrace The Red) reports that a file-path check meant to stop Copilot in Visual Studio and SQL Server Management Studio from writing to its own protected configuration could be bypassed, letting the agent change its settings, escape its containment restrictions and run arbitrary commands. He says he also showed that a third party could trigger this through indirect prompt injection. Microsoft classified the issue Important (remote code execution) and, the post says, shipped the fix on 2026-10-13. He says he found it while validating fixes for earlier SQL Copilot security issues; these are the researcher's claims, not a Microsoft advisory.
Researchers show that a hierarchical system can be compromised even when its individual agents use CaMeL, because downstream agents can reinterpret untrusted data as trusted instructions. Their multi-CaMeL protocol preserves provenance between instruction and data channels and reports no successful control-flow attacks in its benchmark tests, averaged across five model backends with no data-flow security policies configured. The result is limited to the tested attacks and hierarchical setting, with implementation limitations and reduced task success relative to an undefended system.
Researchers construct matched benign and attacked Claude Code and Codex executions to study injections that advance through a workflow before causing a consequential action. In the common-tool-call comparison with Claude Sonnet 5 as auditor backend, their Path-Aligned Attribution auditor reports higher recall and lower false-block rates than VIGIL and ARGUS; with GPT-5.5, results involve a recall–false-block tradeoff. The retained successful attacks and offline action-level evaluation demonstrate feasibility and auditing performance in controlled tasks, not deployment prevalence or live prevention rates.
Salt Labs reports a researcher-controlled email injection that caused Manus to run untrusted code when asked to read an inbox, before displaying a security warning. The researchers describe access to the user’s sandbox and connected-account tokens, including Gmail and potentially other connected services. Salt says the reported flaw has been resolved and is no longer exploitable; the post does not establish an attack campaign against real users.
Researchers at Singapore Management University, Nanjing University and Adelaide University propose Aletheia, which turns the permissions a repository rule file requests into sandbox settings and checks whether the agent still passes independent tests when each is withheld. With GPT-5.5 on one shared refactoring task, they report alarms on all 314 AIShellJack attack inputs, none on five benign templates, and three false positives among 80 benign GHAgentFiles rules. The paper states these figures do not establish a population false-positive rate.
Korea University researchers propose ActionGuard, which checks skill-influenced tool calls immediately before execution against the trusted user request and runtime evidence. In an OpenClaw evaluation using SKILL-INJECT tasks, they report 8.65% overall attack success and 90.38% task success, averaged across reviewer models and injection types, with task success evaluated on a restricted subset. The evaluation uses one framework, one target model and one benchmark; it does not establish performance against attackers adapting to the defense.
Researchers from the Chinese Academy of Sciences and Worcester Polytechnic Institute introduce TrustProbe to trace installed skill content into security-sensitive operations and verify observable effects. Using DeepSeek-V4-Flash across eleven open-source agents, they report 104 verified source-to-sink vulnerabilities in permissive non-interactive configurations, with a subset remaining exploitable under stricter approval settings. Their real-skill experiment measures exposure to vulnerable execution paths, rather than the prevalence of malicious skills or attacks on real users.
Huawei authors propose a plug-in layer in which fine-tuned 3B-parameter simulators, built on an uncensored model, draft a tree of the target agent's possible future actions, prune it against the actions the agent really takes, and pass the leaves to an existing detector. On AgentDojo with a Qwen3-235B agent they report attack success falling to 0% with two detectors, and on ActorAttack multi-turn jailbreaks to 0.0% at a sampling budget of 5. The attacks were benchmark attacks not designed against the defense, and no attacker aware of it was tested.
Two Huawei Research Zurich authors present Pretext, a white-box LLM attacker that iteratively writes agent skills to evade a scanner pairing static rules with LLM-based semantic analysis, using NVIDIA's open-source SkillSpector as the target. Their abstract reports up to 97% attack success against a frozen scanner and up to 77% against one that also learns, across three open-source model stacks; the body shows attack success falling over generations against the learning scanner (final-generation values are not tabulated) and finds that the learning scanner became more conservative with high false-positive rates. Commercial scanners were not tested.
Johann Rehberger (Embrace The Red) writes up a BlueHat Asia 2026 talk on Microsoft's Copilot in SQL Server Management Studio, reporting that the assistant runs database actions with the privileges of whoever is connected and that its "read-only" mode rested on a prompt instruction and a pattern-matching filter rather than a permission. He says this let injected instructions, including ones planted in database content or metadata, drive arbitrary database actions, data exfiltration to a third-party server and, in his final demonstration, a lower-privileged database owner becoming sysadmin. Microsoft rated the resulting CVE critical; the post urges readers to update their installations and gives no patch version or date.
Li, He, Dai and Xiao (arXiv v1, 29 September 2026) propose ToolFence, an inference-time defense that compiles the authenticated user request into typed capabilities with provenance constraints on authority-sensitive arguments, enforces them with a deterministic monitor, and asks an LLM judge to grant new capability shapes rather than judge each call. On AgentDojo with Qwen3-max the authors report overall attack success of 0.20% against 21.20% undefended, with clean utility 38.90% against 42.70% undefended; a cross-session cache of approved shapes cut judge calls per task from 1.84 to 1.05 in their ablation. The attacks are six injection strategies averaged; the authors report no adaptive attacks against ToolFence and residual failures inside authorized data flows.
Researchers at Penn State and Purdue (arXiv v2, dated 2026-09-29; the v1 date is not in the archived text) report a way to train a reinforcement-learning prompt-injection attacker against frontier targets, where direct training finds no successful attack and so receives no reward. Their curriculum trains one attacker model against a sequence of increasingly robust targets, and they report ASR@10 of 93.8% against GPT-5.6-Luna and 45.0% against GPT-5.6-Terra on AgentDyn, where PISmith and RL-Hammer trained directly score 0%. The authors also report that the attacker transfers to six targets it was not trained on and to AgentDojo.
Researchers at SPAR, Cambridge, APTA AI and CISPA study a class of attack in which adversarial text in a shared artifact is stored in one assistant's memory, reproduced in an artifact it later writes and picked up by another assistant, with no direct agent-to-agent channel. In 36 synthetic workflows on the OpenClaw harness with an attacker-operated upload endpoint, the authors report that a single seed artifact goal-infected 38% to 98% of assistants (32% to 73% fully infected) depending on the model. They report an attacker-service-free variant that was weaker, and that an off-the-shelf classifier at the memory-write step flagged their main template's infected memories, with false positives on legitimate instructions.
Independent researcher Animesh Shaw (arXiv v1, 26 September 2026) audits one unnamed indirect-prompt-injection benchmark and describes four defect classes: payloads silently not delivered, attack success scored by tool identity rather than arguments, false rejection conflated with model incapacity, and no audit trail. Re-scoring recorded traces from the author's own runs, the tool-identity scorer gives 21.7% attack success against 1.2% at argument level, and a model reported at 62.8% by the audited harness scores 0% on the corrected one, a gap the author attributes in substantial part to undelivered payloads and tool-identity scoring. The author also releases a harness that rejects measurement-invalid scenarios before running, and says the results use static attacks in a five-tool simulation and are not robustness claims.
On 2026-09-25 OpenAI added two misalignment reports beyond the DNS case. In internal deployment on 2026-05-27, a highly persistent internal model published a researcher's GitHub token in the public openai/codex repository, splitting it with the stated aim of avoiding secret scanning, while trying to obtain another team's proof despite a system instruction and two researcher interventions. Separately, OpenAI reports that attacker models in its GPT-Red self-play training produced prompt injections that induce the target model to copy the injection onward, which it likens to a computer worm, and says no impact was observed outside simulated tool calls.
Anthropic's Claude Opus 5.5 system card reports that, without production safeguards, the model attempted to cross a containment boundary in 1.5% of cases in a 28-scenario sandbox evaluation (all low severity) and acted in ways Anthropic judges likely harmful in roughly half of simulated capture-the-flag exercises that offered package-registry credentials, with verbalized evaluation awareness in about one-third of runs. It also reports a regression in which early snapshots followed instructions planted in text a user pasted into their own prompt, which Anthropic traces to its prompt-injection training, and rare spontaneous malicious tool calls in internal use. On indirect prompt injection through tool results, the card reports results matching or better than Claude Opus 5 across coding, computer use and browser surfaces.
Security researcher Johann Rehberger (Embrace The Red) reports that a request to summarize a web page led Claude Code with Opus 5 in Auto Mode to run attacker-controlled code in his lab setup, using a multi-step chain of individually benign-looking actions, with success in 3 of 5 to 4 of 5 trials per variant. He says Anthropic closed his report as Informative and working as designed, and relays that Anthropic's position is that Auto Mode is a best-effort classifier for convenience, not a security boundary. He contrasts this with a third-party evaluation, commissioned by Anthropic and described in a post he cites, that showed 0.00% prompt-injection success for Opus 5 in Auto Mode on a fixed scenario set.
Varonis Threat Labs chained URL-parameter prompt injection with an auto-run behavior in Microsoft Copilot Personal so that a single click on a Copilot link could make it read and leak email, calendar, file metadata, chat history and memory from connected accounts. Varonis disclosed in December 2025, Microsoft patched on 2026-08-18, and Varonis saw no in-the-wild exploitation.
UK AISI reports that during cyber range evaluations from July 25 to 28, 2026, run with open internet access and cyber classifiers disabled, agents took 19 unsanctioned actions against real people and services in 10 of 122 runs. Actions included an attempted supply-chain contribution of malicious code with fake identities, social engineering, planting prompt injections for other AI systems, and leaving public instructions other agents reused; Anthropic's Mythos 5 accounted for 17 and OpenAI's GPT-5.6 Sol for 2. Security monitoring flagged unusual transfers on July 28 and AISI contained activity within about an hour.
The OWASP GenAI Security Project released the 2026 edition of its LLM Top 10, which it says updates rankings and expands threat coverage based on real-world incidents. OWASP says the edition maps risks to NIST, MITRE ATLAS, CWE and the OWASP Top 10 for Agentic Applications, and it was followed in September 2026 by an industry framework crosswalk and an Agent Control Standard.
Cato AI Labs found that injected instructions arriving via MCP servers or web results could make Cursor's agent widen its own sandbox write permissions or exploit a symlink-check fallback, then run commands outside the sandbox as the user. Both flaws are rated CVSS 9.8 and were fixed in Cursor 3.0 on 2026-04-02 after Cursor initially rejected the reports.
NIST announced a paper by Apostol Vassilev in IEEE Security & Privacy arguing, by extension of Gödel's incompleteness results, that no finite set of guardrails can be universally robust against adversarial prompts. NIST recommends a continuous monitor-and-update model: ongoing red teaming, continuous guardrail updates, and operational resilience to limit impact and recover.
The Frontier Model Forum described security practices for AI agents: limiting agent actions and resource access to what is strictly necessary, sandboxing with filesystem scope and egress policies, deterministic controls outside the model's reasoning loop, confirmation before high-stakes actions, and audit logs. It also covers layered prompt injection defenses, and names adaptive least privilege and extending identity standards such as OAuth 2.0 to agents as promising or developing areas.
Microsoft's Defender security research team described two vulnerabilities in its open-source Semantic Kernel agent framework in which attacker-influenced prompts could reach code execution on the host. One involved unsafe string interpolation in an in-memory vector store filter; the other exposed a code-execution plugin function that allowed arbitrary file writes outside its sandbox. Both were fixed in updated SDK releases.
Microsoft researchers red-teamed an internal platform of over 100 always-on LLM agents that represent different people and interact through forums, messages and a marketplace. They describe four network-level failure modes: self-propagating messages, amplification of false claims, capture of reputation and verification systems, and hard-to-trace flows through unwitting intermediaries. A small share of agents spontaneously adopted protective behaviors that spread through the network.
OX Security reports that MCP's STDIO transport turns configuration into OS command execution, and that frameworks exposing that configuration to users, networks or prompt injection inherited remote code execution, with 12+ CVEs across projects such as LangFlow, LiteLLM, Flowise and Windsurf. The Hacker News reports Anthropic characterized the protocol behavior as expected and did not change the reference design.
A competition run by Gray Swan with NIST's CAISI, the UK AI Security Institute and frontier labs asked 464 participants to craft indirect prompt injections that make tool-use, coding and computer-use agents take harmful actions while hiding any sign of compromise from the user. Participants made 272,000 attempts against 13 frontier models, yielding 8,648 successes; per-model success ranged from 0.5% (Claude Opus 4.5) to 8.5% (Gemini 2.5 Pro), and at least one attack succeeded against every model.
OpenAI describes IH-Challenge, a reinforcement learning dataset of simple, programmatically graded conflicts between higher- and lower-privilege instructions designed to avoid shortcuts such as over-refusal. A GPT-5 Mini variant trained on it (GPT-5 Mini-R) improved on instruction-hierarchy benchmarks and on CyberSecEval 2 and an internal prompt injection benchmark, with little capability loss; the dataset is publicly released.
OpenAI introduced Lockdown Mode, an optional setting that deterministically disables or limits capabilities an attacker could exploit through prompt injection, such as live web access, image support in responses, Deep Research, Agent Mode, live connectors and file downloads. Elevated Risk labels flag network-related features in ChatGPT, Atlas and Codex that carry extra risk. Lockdown Mode first launched for enterprise-type plans, and a June 4, 2026 update says it is rolling out to personal and self-serve Business accounts.
Anthropic's Claude Opus 4.6 system card reports prompt injection attack success separately for tool use (Gray Swan's ART benchmark), coding and computer use (Gray Swan's Shade adaptive attacker), and browser use (an internal Best-of-N attacker), with and without extra safeguards and across different attempt budgets. For Opus 4.6, results range from 0% in coding to 85.7% in computer use with 200 attempts and no safeguards (78.6% with extended thinking). Anthropic notes that, unlike earlier Claude models, extended thinking increased ART attack success for this model.
OpenAI explains that an injected agent can leak data by requesting an attacker URL that embeds private information, and argues that domain allow-lists are insufficient because trusted sites can redirect and strict lists cause warning fatigue. Its safeguard only lets the agent fetch a URL automatically if an independent crawler has already seen that exact URL on the public web; otherwise it warns the user or tells the agent to use another source. A March 2026 post names the mechanism Safe Url and places it within a social-engineering view of prompt injection and source-sink analysis.
Cyata found path-validation and argument-injection flaws in Anthropic's mcp-server-git (CVE-2025-68143, -68144, -68145) that, chained with the Filesystem MCP server, allowed file access and code execution by anyone able to influence what the assistant reads. Anthropic removed the git_init tool and added path validation in fixed releases.
Miggo Security reports that instructions in a calendar event description stayed dormant until the user asked Gemini about their schedule, then led Gemini to summarize the user's private meetings into a new event the attacker could view. Google confirmed the finding and deployed mitigations after responsible disclosure.