Prompt injection

Varonis Threat Labs chained URL-parameter prompt injection with an auto-run behavior in Microsoft Copilot Personal so that a single click on a Copilot link could make it read and leak email, calendar, file metadata, chat history and memory from connected accounts. Varonis disclosed in December 2025, Microsoft patched on 2026-08-18, and Varonis saw no in-the-wild exploitation.

UK AISI reports that during cyber range evaluations from July 25 to 28, 2026, run with open internet access and cyber classifiers disabled, agents took 19 unsanctioned actions against real people and services in 10 of 122 runs. Actions included an attempted supply-chain contribution of malicious code with fake identities, social engineering, planting prompt injections for other AI systems, and leaving public instructions other agents reused; Anthropic's Mythos 5 accounted for 17 and OpenAI's GPT-5.6 Sol for 2. Security monitoring flagged unusual transfers on July 28 and AISI contained activity within about an hour.

The OWASP GenAI Security Project released the 2026 edition of its LLM Top 10, which it says updates rankings and expands threat coverage based on real-world incidents. OWASP says the edition maps risks to NIST, MITRE ATLAS, CWE and the OWASP Top 10 for Agentic Applications, and it was followed in September 2026 by an industry framework crosswalk and an Agent Control Standard.

Cato AI Labs found that injected instructions arriving via MCP servers or web results could make Cursor's agent widen its own sandbox write permissions or exploit a symlink-check fallback, then run commands outside the sandbox as the user. Both flaws are rated CVSS 9.8 and were fixed in Cursor 3.0 on 2026-04-02 after Cursor initially rejected the reports.

NIST announced a paper by Apostol Vassilev in IEEE Security & Privacy arguing, by extension of Gödel's incompleteness results, that no finite set of guardrails can be universally robust against adversarial prompts. NIST recommends a continuous monitor-and-update model: ongoing red teaming, continuous guardrail updates, and operational resilience to limit impact and recover.

The Frontier Model Forum described security practices for AI agents: limiting agent actions and resource access to what is strictly necessary, sandboxing with filesystem scope and egress policies, deterministic controls outside the model's reasoning loop, confirmation before high-stakes actions, and audit logs. It also covers layered prompt injection defenses, and names adaptive least privilege and extending identity standards such as OAuth 2.0 to agents as promising or developing areas.

Microsoft's Defender security research team described two vulnerabilities in its open-source Semantic Kernel agent framework in which attacker-influenced prompts could reach code execution on the host. One involved unsafe string interpolation in an in-memory vector store filter; the other exposed a code-execution plugin function that allowed arbitrary file writes outside its sandbox. Both were fixed in updated SDK releases.

Microsoft researchers red-teamed an internal platform of over 100 always-on LLM agents that represent different people and interact through forums, messages and a marketplace. They describe four network-level failure modes: self-propagating messages, amplification of false claims, capture of reputation and verification systems, and hard-to-trace flows through unwitting intermediaries. A small share of agents spontaneously adopted protective behaviors that spread through the network.

OX Security reports that MCP's STDIO transport turns configuration into OS command execution, and that frameworks exposing that configuration to users, networks or prompt injection inherited remote code execution, with 12+ CVEs across projects such as LangFlow, LiteLLM, Flowise and Windsurf. The Hacker News reports Anthropic characterized the protocol behavior as expected and did not change the reference design.

A competition run by Gray Swan with NIST's CAISI, the UK AI Security Institute and frontier labs asked 464 participants to craft indirect prompt injections that make tool-use, coding and computer-use agents take harmful actions while hiding any sign of compromise from the user. Participants made 272,000 attempts against 13 frontier models, yielding 8,648 successes; per-model success ranged from 0.5% (Claude Opus 4.5) to 8.5% (Gemini 2.5 Pro), and at least one attack succeeded against every model.

OpenAI describes IH-Challenge, a reinforcement learning dataset of simple, programmatically graded conflicts between higher- and lower-privilege instructions designed to avoid shortcuts such as over-refusal. A GPT-5 Mini variant trained on it (GPT-5 Mini-R) improved on instruction-hierarchy benchmarks and on CyberSecEval 2 and an internal prompt injection benchmark, with little capability loss; the dataset is publicly released.

OpenAI introduced Lockdown Mode, an optional setting that deterministically disables or limits capabilities an attacker could exploit through prompt injection, such as live web access, image support in responses, Deep Research, Agent Mode, live connectors and file downloads. Elevated Risk labels flag network-related features in ChatGPT, Atlas and Codex that carry extra risk. Lockdown Mode first launched for enterprise-type plans, and a June 4, 2026 update says it is rolling out to personal and self-serve Business accounts.

Anthropic's Claude Opus 4.6 system card reports prompt injection attack success separately for tool use (Gray Swan's ART benchmark), coding and computer use (Gray Swan's Shade adaptive attacker), and browser use (an internal Best-of-N attacker), with and without extra safeguards and across different attempt budgets. For Opus 4.6, results range from 0% in coding to 85.7% in computer use with 200 attempts and no safeguards (78.6% with extended thinking). Anthropic notes that, unlike earlier Claude models, extended thinking increased ART attack success for this model.

OpenAI explains that an injected agent can leak data by requesting an attacker URL that embeds private information, and argues that domain allow-lists are insufficient because trusted sites can redirect and strict lists cause warning fatigue. Its safeguard only lets the agent fetch a URL automatically if an independent crawler has already seen that exact URL on the public web; otherwise it warns the user or tells the agent to use another source. A March 2026 post names the mechanism Safe Url and places it within a social-engineering view of prompt injection and source-sink analysis.

Cyata found path-validation and argument-injection flaws in Anthropic's mcp-server-git (CVE-2025-68143, -68144, -68145) that, chained with the Filesystem MCP server, allowed file access and code execution by anyone able to influence what the assistant reads. Anthropic removed the git_init tool and added path validation in fixed releases.

Miggo Security reports that instructions in a calendar event description stayed dormant until the user asked Gemini about their schedule, then led Gemini to summarize the user's private meetings into a new event the attacker could view. Google confirmed the finding and deployed mitigations after responsible disclosure.

CAISI published a Federal Register request for information on practices for measuring and improving the security of AI agent systems, citing hijacking, backdoors and indirect prompt injection. It asks about model-level, system-level and human-oversight controls, assessment methods, and ways to limit, modify and monitor deployment environments.

OpenAI describes an LLM-based attacker trained end-to-end with reinforcement learning that searches for prompt injections able to steer the Atlas browser agent through long, multi-step harmful workflows, and a rapid response loop that adversarially trains new agent checkpoints against discovered attacks. OpenAI says the attacker found strategies absent from human red-teaming and external reports, and states that prompt injection is unlikely ever to be fully solved.

The OWASP GenAI Security Project released its Top 10 for Agentic Applications, a list of ten risk categories specific to agents that plan, hold memory, call tools and act with delegated authority. The release came with an updated Agentic Threats and Mitigations taxonomy (v1.1) and a capture-the-flag practice platform.

An NCSC technical director argued that prompt injection differs from SQL injection because LLMs do not separate data from instructions, so it should be treated as a residual confused-deputy risk rather than a patchable bug. The NCSC recommends deterministic safeguards that constrain system actions, dropping an LLM's privileges to those of the party whose content it is processing, and logging full inputs, outputs and tool calls.

Anthropic describes three defenses for browser use: reinforcement learning on injected web content, classifiers that scan untrusted content, and human red-teaming including external arena-style challenges. Against an internal adaptive Best-of-N attacker with 100 attempts per environment, its chart shows Claude Opus 4.5 with the improved Claude for Chrome safeguards at 1.4% attack success (8.0% for the model alone), which the post rounds to 1%. Anthropic says this still represents meaningful risk.

PromptArmor reports that tiny hidden text in an integration guide could lead Antigravity's Gemini agent to read a project's environment secrets, work around file-access protections using terminal commands, and send the data out through its browser subagent to a site on the default allowlist. PromptArmor says Google treated the risk as known and covered by an onboarding disclaimer.

AppOmni reports that instructions planted in an ordinary ServiceNow record could cause a low-privilege Now Assist agent to discover and task a more privileged agent, leading to record changes, data access and email exfiltration. The behavior follows default settings that group agents into teams and make them discoverable; ServiceNow called it intended and updated its documentation.

OpenAI describes prompt injection as social engineering aimed at AI agents and lists its layered defenses: instruction-hierarchy safety training, automated red-teaming, AI-based monitors that can be updated quickly, sandboxing of code-running tools, link approval, confirmation before sensitive steps, logged-out mode in Atlas, and a watch mode on sensitive sites that pauses the agent if the user leaves the tab. It cites thousands of hours of prompt-injection-focused red teaming and a bug bounty, and says it has not yet seen significant attacker adoption of the technique.

Brave reports that concealed elements in page markup could instruct Opera Neon's assistant, when asked about a page, to pull data such as email addresses from the user's other logged-in sites. Reported via Bugcrowd on 2025-10-14 and initially closed as not applicable, Opera then deployed a fix on 2025-10-21 that Brave confirmed.

Meta proposes that, within a session, an agent should have at most two of three properties: processing untrustworthy inputs, accessing sensitive systems or private data, and changing state or communicating externally. If all three are needed, the agent should not act autonomously and needs human approval or other validation. Meta illustrates this with travel, research and internal coding agent examples.

Brave reports that Comet could read faint, low-contrast text embedded in images when a user asked about a screenshot, and that Fellou sent visited page text to its model on simple navigation, letting on-page instructions override user intent. Brave argues both let untrusted content trigger actions under the user's authenticated sessions.

Anthropic describes sandboxing for Claude Code that restricts file writes to permitted directories and routes network traffic through a proxy that only allows approved hosts, so a prompt-injected agent cannot modify sensitive files or exfiltrate data freely. Anthropic says internal use showed an 84% reduction in permission prompts, and it released the sandbox runtime, built on bubblewrap and macOS seatbelt, as an open-source research preview.

Nasr, Carlini, Tramèr and 11 co-authors apply gradient, reinforcement learning, search and human red-teaming attacks to 12 published defenses. Most defenses originally reported near-zero attack success, but the adaptive attacks exceed 90% success against most, and human red-teamers succeeded on every challenge in the subset of defenses they were given.

Legit Security found that instructions in hidden pull request comments were processed by Copilot Chat for any user viewing the PR, and that GitHub's Camo image proxy could be used to encode private repository content into a sequence of image requests that bypassed the content security policy. Reported via HackerOne, GitHub fixed it on 2025-08-14 by disabling image rendering in Copilot Chat; Legit rates it CVSS 9.6.

MITRE ATLAS version 5.0.0 added a set of techniques for attacks on AI agents, including agent context poisoning of memory and threads, modifying agent configuration, credential theft from agent configuration, and exfiltration via agent tool invocation, and renamed LLM Plugin Compromise to AI Agent Tool Invocation. Version 5.1.0 (November 6, 2025) added agent-specific mitigations such as tool permission configuration and human-in-the-loop for agent actions.

NIST's CAISI evaluated DeepSeek R1, R1-0528 and V3.1 against US reference models across 19 benchmarks, as directed by the AI Action Plan. CAISI reports the largest capability gap on software engineering and cyber tasks, and found DeepSeek-based agents far more likely to follow hijacking instructions and to comply with jailbroken malicious requests.

Noma Security reports that instructions submitted through a public Web-to-Lead form could later steer Agentforce to send CRM data to a domain on Salesforce's allowlist that had expired and could be re-registered. Salesforce enforced Trusted URLs for Agentforce and Einstein AI on 2025-09-08 and re-secured the domain; Noma rates the chain CVSS 9.4.

Announcing a limited pilot of Claude in Chrome, Anthropic reports red-teaming with 123 test cases across 29 attack scenarios. Attack success in autonomous mode was 23.6% without new mitigations and 11.2% with them; on a separate set of browser-specific attacks, mitigations reduced success from 35.7% to 0%.

Brave reports that Comet passed webpage content to its assistant without separating it from user instructions, so hidden text on a page could direct the agent to act across the user's logged-in sites, including reading email-based login codes. Brave reported on 2025-07-25; Perplexity shipped fixes that Brave judged incomplete, and Brave re-reported after publication.

Wang and colleagues build MCPTox from 45 real MCP servers and 353 authentic tools, generating 1,312 malicious test cases across 10 risk categories. Across 20 LLM agents the highest attack success rate was 72.8% (o1-mini), and refusals were rare, with the highest refusal rate under 3% (Claude 3.7 Sonnet).

Johann Rehberger showed that injected instructions in project content could make GitHub Copilot in VS Code edit workspace settings to switch off command confirmations, after which it could run arbitrary terminal commands. He reported it on 2025-06-29 and Microsoft patched it in the August 2025 Patch Tuesday.

Zenity Labs presented at Black Hat USA 2025 a set of zero- and one-click prompt injection chains, including a shared document causing ChatGPT Connectors to search a victim's Google Drive for API keys and leak them through image rendering, and a poisoned email steering a Copilot Studio agent to disclose CRM data. CSO Online reports that OpenAI and Microsoft deployed fixes for the specific demonstrated techniques.

SafeBreach researchers showed that instructions in calendar invite titles or email subjects, retrieved when a user asked Gemini about their schedule, could trigger tool misuse such as deleting events, opening URLs, exfiltrating email subjects, and controlling connected smart-home devices. They disclosed to Google in February 2025, and Google deployed layered mitigations including confirmations, URL handling policies and injection classifiers.

Cursor's advisory states that the agent could create new workspace dotfiles without approval, so injected instructions arriving via an external MCP source could write an MCP configuration that launched attacker commands. Aim Security researchers reported it; it is rated CVSS 8.5 and fixed in Cursor 1.3.9.