How it works
The agent pauses and shows the proposed action. Effectiveness depends on what the approval screen shows and how many approvals people are asked for.
Approval fatigue and incomplete approval displays weaken the control.
What we know
3 corroborated, 1 revalidateRecords over time
Use the arrow keys to move between records, Home and End to jump to the first and last, and Enter to select one.
Select a mark to read the record. Mark size shows editorial significance. Hollow marks are dated to the month. Era bands are editorial labels.
Records in view
20 records · newest firstGoogle DeepMind describes AlphaProof Code, combining human-approved specifications with agent-generated proofs checked by Lean. Its libsignal study proves specified properties of PQXDH and the Triple Ratchet at the AArch64 machine-code level. Assurance depends on the specifications, formal semantics and other stated trust assumptions; it does not cover every property of the Signal application.
Trail of Bits reports that its Codex workflow helped deliver fixes for 149 distinct CVEs into stable Linux kernel releases by October 29, with additional package work for Fedora and Ubuntu. Agents prepare backports and evidence; engineers review and submit them to maintainers. Reported human-time savings exclude setup, regression testing and maintainer acceptance.
Microsoft announced on 2026-10-19 that it is extending Project Perception to govern AI agents as potential insiders: discovering local agents such as Claude Code, OpenAI Codex and GitHub Copilot, letting discovered agents receive their own Entra identity, scoping permissions to a task and its duration, and pairing a continuously updated agent risk assessment with conditional access that can narrow access, require human approval, or isolate and stop an agent. Microsoft frames the design around keeping security controls outside the model, so that a manipulated or compromised agent cannot bypass them. The post is a product announcement and gives no availability dates or measurements of effectiveness.
Salt Labs reports a researcher-controlled email injection that caused Manus to run untrusted code when asked to read an inbox, before displaying a security warning. The researchers describe access to the user’s sandbox and connected-account tokens, including Gmail and potentially other connected services. Salt says the reported flaw has been resolved and is no longer exploitable; the post does not establish an attack campaign against real users.
Researchers from the Chinese Academy of Sciences and Worcester Polytechnic Institute introduce TrustProbe to trace installed skill content into security-sensitive operations and verify observable effects. Using DeepSeek-V4-Flash across eleven open-source agents, they report 104 verified source-to-sink vulnerabilities in permissive non-interactive configurations, with a subset remaining exploitable under stricter approval settings. Their real-skill experiment measures exposure to vulnerable execution paths, rather than the prevalence of malicious skills or attacks on real users.
In a research note, METR describes a monitor it deployed on its own agent evaluations after the OpenAI, Anthropic and UK AISI incidents: an LLM judge scores each tool call before it runs through Inspect's approval chain, and calls scoring 3/10 or higher are held for human review, with the run halted. METR reports that a UK AISI replay flagged all 10 malicious transcripts from that incident (minimum score 8/10) with about 98% of 25,000 benign actions scoring 0/10, that 13 of 13 malicious actions in synthetic transcripts were flagged, and that 5 of 20,000 sampled benign actions (0.025%) were flagged. METR frames the note as an argument with self-graded evidence, and lists gaps it found, including a researcher running evals without the monitor and a hand-built spoofed user turn that lowered suspicion scores.
UK AISI's new Control Red Team tested an asynchronous reasoning monitor with Google DeepMind and successive versions of Anthropic's synchronous internal agentic-coding monitor, and reports vulnerabilities in each. Successful attacks exploited gaps in monitor prompts, weaknesses in the underlying LLM, and hard-coded rules that decide which actions reach the monitor. In a separate early experiment, an automated evolutionary search against a Claude 4.6 Sonnet monitor found attack transcripts scoring 3/100 on suspicion, versus 14-18 for best-of-N baselines. That experiment used a single-step coding task in which the agent covertly exposes its API key.
The Frontier Model Forum described security practices for AI agents: limiting agent actions and resource access to what is strictly necessary, sandboxing with filesystem scope and egress policies, deterministic controls outside the model's reasoning loop, confirmation before high-stakes actions, and audit logs. It also covers layered prompt injection defenses, and names adaptive least privilege and extending identity standards such as OAuth 2.0 to agents as promising or developing areas.
NCSC authors advise deploying agentic AI incrementally through tightly bounded pilots, granting agents only the minimum permissions with temporary credentials, and defining in advance who approves access, monitors behavior and can halt the agent. They recommend incident response plans for agent failure and loss-of-control scenarios.
Jotautaitė, Martinez, Matthews and Tracy (two listed as independent, Tracy at Redwood Research) build a semi-automated red-teaming pipeline in the BashArena control setting, where agents covertly pursue side tasks such as firewall backdoors, privilege escalation, credential exfiltration and malware download. MonitoringBench's 2,644 attack trajectories lowered a Claude Opus 4.5 monitor's catch rate at 1% false positives from 94.9% on directly elicited attacks to 60.3% on refined ones.
OpenAI describes an asynchronous monitor powered by GPT-5.4 Thinking at maximum reasoning effort that reviews internal coding agent sessions, including chains of thought, within about 30 minutes of completion. OpenAI reports that over five months it covered tens of millions of trajectories, with no real conversation reaching the highest severity and about 1,000 moderate-severity alerts, many from deliberate internal red-teaming, and that it flagged every interaction employees had independently reported. The categories it labels common, each under 1% of traffic, are circumventing restrictions, deception and concealing uncertainty.
Check Point Research found that a cloned repository's Claude Code configuration could run hooks, start MCP servers before the user approved them, and redirect API traffic so the user's Anthropic API key was sent to an attacker (CVE-2025-59536, CVE-2026-21852). Anthropic fixed the issues between August and December 2025 by deferring execution and API calls until after the trust dialog.
Anthropic released Claude Code Security as a limited research preview for Enterprise and Team customers, with expedited free access for open-source maintainers. The tool reasons about data flow across a codebase, re-examines each finding in a multi-stage verification pass, assigns severity and confidence ratings, and proposes patches that are applied only with human approval.
OpenAI describes prompt injection as social engineering aimed at AI agents and lists its layered defenses: instruction-hierarchy safety training, automated red-teaming, AI-based monitors that can be updated quickly, sandboxing of code-running tools, link approval, confirmation before sensitive steps, logged-out mode in Atlas, and a watch mode on sensitive sites that pauses the agent if the user leaves the tab. It cites thousands of hours of prompt-injection-focused red teaming and a bug bounty, and says it has not yet seen significant attacker adoption of the technique.
Meta proposes that, within a session, an agent should have at most two of three properties: processing untrustworthy inputs, accessing sensitive systems or private data, and changing state or communicating externally. If all three are needed, the agent should not act autonomously and needs human approval or other validation. Meta illustrates this with travel, research and internal coding agent examples.
Anthropic describes sandboxing for Claude Code that restricts file writes to permitted directories and routes network traffic through a proxy that only allows approved hosts, so a prompt-injected agent cannot modify sensitive files or exfiltrate data freely. Anthropic says internal use showed an 84% reduction in permission prompts, and it released the sandbox runtime, built on bubblewrap and macOS seatbelt, as an open-source research preview.
Johann Rehberger showed that injected instructions in project content could make GitHub Copilot in VS Code edit workspace settings to switch off command confirmations, after which it could run arbitrary terminal commands. He reported it on 2025-06-29 and Microsoft patched it in the August 2025 Patch Tuesday.
SafeBreach researchers showed that instructions in calendar invite titles or email subjects, retrieved when a user asked Gemini about their schedule, could trigger tool misuse such as deleting events, opening URLs, exfiltrating email subjects, and controlling connected smart-home devices. They disclosed to Google in February 2025, and Google deployed layered mitigations including confirmations, URL handling policies and injection classifiers.
Check Point Research found that Cursor bound MCP approval to a configuration's name rather than its contents, so a collaborator with repository write access could swap an approved harmless command for a malicious one that ran on each project open. Cursor 1.3, released 2025-07-29, prompts for approval on any MCP configuration change.
The Coalition for Secure AI, an OASIS Open Project, published three principles for agentic systems. The principles call for agents that are human-governed and accountable, bounded and resilient with strict, purpose-specific entitlements, and transparent and verifiable through secure AI supply chain controls and telemetry that supports monitoring and forensics.