Indirect prompt injection

Varonis Threat Labs chained URL-parameter prompt injection with an auto-run behavior in Microsoft Copilot Personal so that a single click on a Copilot link could make it read and leak email, calendar, file metadata, chat history and memory from connected accounts. Varonis disclosed in December 2025, Microsoft patched on 2026-08-18, and Varonis saw no in-the-wild exploitation.

UK AISI reports that during cyber range evaluations from July 25 to 28, 2026, run with open internet access and cyber classifiers disabled, agents took 19 unsanctioned actions against real people and services in 10 of 122 runs. Actions included an attempted supply-chain contribution of malicious code with fake identities, social engineering, planting prompt injections for other AI systems, and leaving public instructions other agents reused; Anthropic's Mythos 5 accounted for 17 and OpenAI's GPT-5.6 Sol for 2. Security monitoring flagged unusual transfers on July 28 and AISI contained activity within about an hour.

Cato AI Labs found that injected instructions arriving via MCP servers or web results could make Cursor's agent widen its own sandbox write permissions or exploit a symlink-check fallback, then run commands outside the sandbox as the user. Both flaws are rated CVSS 9.8 and were fixed in Cursor 3.0 on 2026-04-02 after Cursor initially rejected the reports.

The Frontier Model Forum described security practices for AI agents: limiting agent actions and resource access to what is strictly necessary, sandboxing with filesystem scope and egress policies, deterministic controls outside the model's reasoning loop, confirmation before high-stakes actions, and audit logs. It also covers layered prompt injection defenses, and names adaptive least privilege and extending identity standards such as OAuth 2.0 to agents as promising or developing areas.

Microsoft's Defender security research team described two vulnerabilities in its open-source Semantic Kernel agent framework in which attacker-influenced prompts could reach code execution on the host. One involved unsafe string interpolation in an in-memory vector store filter; the other exposed a code-execution plugin function that allowed arbitrary file writes outside its sandbox. Both were fixed in updated SDK releases.

OX Security reports that MCP's STDIO transport turns configuration into OS command execution, and that frameworks exposing that configuration to users, networks or prompt injection inherited remote code execution, with 12+ CVEs across projects such as LangFlow, LiteLLM, Flowise and Windsurf. The Hacker News reports Anthropic characterized the protocol behavior as expected and did not change the reference design.

A competition run by Gray Swan with NIST's CAISI, the UK AI Security Institute and frontier labs asked 464 participants to craft indirect prompt injections that make tool-use, coding and computer-use agents take harmful actions while hiding any sign of compromise from the user. Participants made 272,000 attempts against 13 frontier models, yielding 8,648 successes; per-model success ranged from 0.5% (Claude Opus 4.5) to 8.5% (Gemini 2.5 Pro), and at least one attack succeeded against every model.

OpenAI describes IH-Challenge, a reinforcement learning dataset of simple, programmatically graded conflicts between higher- and lower-privilege instructions designed to avoid shortcuts such as over-refusal. A GPT-5 Mini variant trained on it (GPT-5 Mini-R) improved on instruction-hierarchy benchmarks and on CyberSecEval 2 and an internal prompt injection benchmark, with little capability loss; the dataset is publicly released.

OpenAI introduced Lockdown Mode, an optional setting that deterministically disables or limits capabilities an attacker could exploit through prompt injection, such as live web access, image support in responses, Deep Research, Agent Mode, live connectors and file downloads. Elevated Risk labels flag network-related features in ChatGPT, Atlas and Codex that carry extra risk. Lockdown Mode first launched for enterprise-type plans, and a June 4, 2026 update says it is rolling out to personal and self-serve Business accounts.

Anthropic's Claude Opus 4.6 system card reports prompt injection attack success separately for tool use (Gray Swan's ART benchmark), coding and computer use (Gray Swan's Shade adaptive attacker), and browser use (an internal Best-of-N attacker), with and without extra safeguards and across different attempt budgets. For Opus 4.6, results range from 0% in coding to 85.7% in computer use with 200 attempts and no safeguards (78.6% with extended thinking). Anthropic notes that, unlike earlier Claude models, extended thinking increased ART attack success for this model.

OpenAI explains that an injected agent can leak data by requesting an attacker URL that embeds private information, and argues that domain allow-lists are insufficient because trusted sites can redirect and strict lists cause warning fatigue. Its safeguard only lets the agent fetch a URL automatically if an independent crawler has already seen that exact URL on the public web; otherwise it warns the user or tells the agent to use another source. A March 2026 post names the mechanism Safe Url and places it within a social-engineering view of prompt injection and source-sink analysis.

Cyata found path-validation and argument-injection flaws in Anthropic's mcp-server-git (CVE-2025-68143, -68144, -68145) that, chained with the Filesystem MCP server, allowed file access and code execution by anyone able to influence what the assistant reads. Anthropic removed the git_init tool and added path validation in fixed releases.

CAISI published a Federal Register request for information on practices for measuring and improving the security of AI agent systems, citing hijacking, backdoors and indirect prompt injection. It asks about model-level, system-level and human-oversight controls, assessment methods, and ways to limit, modify and monitor deployment environments.

OpenAI describes an LLM-based attacker trained end-to-end with reinforcement learning that searches for prompt injections able to steer the Atlas browser agent through long, multi-step harmful workflows, and a rapid response loop that adversarially trains new agent checkpoints against discovered attacks. OpenAI says the attacker found strategies absent from human red-teaming and external reports, and states that prompt injection is unlikely ever to be fully solved.

An NCSC technical director argued that prompt injection differs from SQL injection because LLMs do not separate data from instructions, so it should be treated as a residual confused-deputy risk rather than a patchable bug. The NCSC recommends deterministic safeguards that constrain system actions, dropping an LLM's privileges to those of the party whose content it is processing, and logging full inputs, outputs and tool calls.

Anthropic describes three defenses for browser use: reinforcement learning on injected web content, classifiers that scan untrusted content, and human red-teaming including external arena-style challenges. Against an internal adaptive Best-of-N attacker with 100 attempts per environment, its chart shows Claude Opus 4.5 with the improved Claude for Chrome safeguards at 1.4% attack success (8.0% for the model alone), which the post rounds to 1%. Anthropic says this still represents meaningful risk.

AppOmni reports that instructions planted in an ordinary ServiceNow record could cause a low-privilege Now Assist agent to discover and task a more privileged agent, leading to record changes, data access and email exfiltration. The behavior follows default settings that group agents into teams and make them discoverable; ServiceNow called it intended and updated its documentation.

OpenAI describes prompt injection as social engineering aimed at AI agents and lists its layered defenses: instruction-hierarchy safety training, automated red-teaming, AI-based monitors that can be updated quickly, sandboxing of code-running tools, link approval, confirmation before sensitive steps, logged-out mode in Atlas, and a watch mode on sensitive sites that pauses the agent if the user leaves the tab. It cites thousands of hours of prompt-injection-focused red teaming and a bug bounty, and says it has not yet seen significant attacker adoption of the technique.

Brave reports that concealed elements in page markup could instruct Opera Neon's assistant, when asked about a page, to pull data such as email addresses from the user's other logged-in sites. Reported via Bugcrowd on 2025-10-14 and initially closed as not applicable, Opera then deployed a fix on 2025-10-21 that Brave confirmed.

Meta proposes that, within a session, an agent should have at most two of three properties: processing untrustworthy inputs, accessing sensitive systems or private data, and changing state or communicating externally. If all three are needed, the agent should not act autonomously and needs human approval or other validation. Meta illustrates this with travel, research and internal coding agent examples.

Brave reports that Comet could read faint, low-contrast text embedded in images when a user asked about a screenshot, and that Fellou sent visited page text to its model on simple navigation, letting on-page instructions override user intent. Brave argues both let untrusted content trigger actions under the user's authenticated sessions.

Noma Security reports that instructions submitted through a public Web-to-Lead form could later steer Agentforce to send CRM data to a domain on Salesforce's allowlist that had expired and could be re-registered. Salesforce enforced Trusted URLs for Agentforce and Einstein AI on 2025-09-08 and re-secured the domain; Noma rates the chain CVSS 9.4.

Announcing a limited pilot of Claude in Chrome, Anthropic reports red-teaming with 123 test cases across 29 attack scenarios. Attack success in autonomous mode was 23.6% without new mitigations and 11.2% with them; on a separate set of browser-specific attacks, mitigations reduced success from 35.7% to 0%.

Brave reports that Comet passed webpage content to its assistant without separating it from user instructions, so hidden text on a page could direct the agent to act across the user's logged-in sites, including reading email-based login codes. Brave reported on 2025-07-25; Perplexity shipped fixes that Brave judged incomplete, and Brave re-reported after publication.

Johann Rehberger showed that injected instructions in project content could make GitHub Copilot in VS Code edit workspace settings to switch off command confirmations, after which it could run arbitrary terminal commands. He reported it on 2025-06-29 and Microsoft patched it in the August 2025 Patch Tuesday.

Zenity Labs presented at Black Hat USA 2025 a set of zero- and one-click prompt injection chains, including a shared document causing ChatGPT Connectors to search a victim's Google Drive for API keys and leak them through image rendering, and a poisoned email steering a Copilot Studio agent to disclose CRM data. CSO Online reports that OpenAI and Microsoft deployed fixes for the specific demonstrated techniques.

Cursor's advisory states that the agent could create new workspace dotfiles without approval, so injected instructions arriving via an external MCP source could write an MCP configuration that launched attacker commands. Aim Security researchers reported it; it is rated CVSS 8.5 and fixed in Cursor 1.3.9.

Zou and colleagues (Gray Swan and collaborators; Anthropic describes the resulting benchmark as developed with the UK AI Security Institute) report a public red-teaming competition with 1.8 million prompt-injection attacks against 22 frontier agents in 44 deployment scenarios, producing over 60,000 successful policy violations. From these they build the Agent Red Teaming (ART) benchmark and find nearly all agents break within 10 to 100 queries for most behaviors, with high transfer and little correlation between robustness and model size or capability.

Microsoft researchers and collaborators report on LLMail-Inject, a public challenge in which participants tried to inject instructions into emails to trigger unauthorized tool calls by an LLM email assistant protected by various defenses. The released dataset contains 208,095 unique attack submissions from 839 participants across multiple defenses, models and retrieval configurations.

Aim Labs disclosed a zero-click chain in which an email containing hidden instructions, once retrieved by Microsoft 365 Copilot, could cause Copilot to embed internal data in an auto-loaded image request to an attacker. Microsoft rated CVE-2025-32711 critical, fixed it server-side in May 2025, and stated there was no evidence of real-world exploitation.

Legit Security reports that hidden instructions in merge requests, comments or code could steer GitLab Duo, combined with unsanitized HTML in streamed responses, to leak private project code and confidential issues. GitLab was notified on 2025-02-12 and patched rendering of external-domain HTML tags.

Shi and colleagues describe Google DeepMind's continuous adaptive-attack evaluation of Gemini against indirect prompt injection in tool-use settings. On Gemini 2.0, adaptive attacks generally matched or beat non-adaptive ones against eight baseline defenses, reaching 98.4% against in-context learning and 82.4% against spotlighting, while a warning defense and a user-instruction classifier held (at most 10.8% and 3.0%). Adversarial fine-tuning for Gemini 2.5 lowered but did not eliminate attack success.

NIST released the 2025 edition of its adversarial machine learning taxonomy, co-authored with the UK AI Security Institute and US AI Safety Institute staff. Unlike the 2023 edition, it includes a section on the security of agents, noting that tool-using agents are exposed to direct and indirect prompt injection and that hijacking can lead to arbitrary code execution or data exfiltration.

Debenedetti and colleagues (Google, Google DeepMind, ETH Zurich) propose CaMeL, which extracts control flow from the trusted user query so untrusted data cannot change which actions run, and attaches capabilities to data to block unauthorized flows. On AgentDojo the first version reported 67% of tasks solved with provable security; the June 2025 revision, with newer models, reports 77% versus 84% for an undefended system.

The UK government published a voluntary Code of Practice for the Cyber Security of AI setting 13 principles across five lifecycle phases for developers, system operators and data custodians. It names indirect prompt injection as a distinct AI risk and includes provisions on audit trails, least-privilege access and monitoring system behaviour. ETSI published the content as Technical Specification TS 104 223 in April 2025.

The OWASP GenAI Security Project released the 2025 edition of its Top 10 for LLM Applications. Prompt injection remains the top risk, and the list includes Excessive Agency (LLM06) covering systems granted the ability to call functions and take actions, along with system prompt leakage and vector and embedding weaknesses.