Human approval gates

UK AISI's new Control Red Team tested an asynchronous reasoning monitor with Google DeepMind and successive versions of Anthropic's synchronous internal agentic-coding monitor, and reports vulnerabilities in each. Successful attacks exploited gaps in monitor prompts, weaknesses in the underlying LLM, and hard-coded rules that decide which actions reach the monitor. In a separate early experiment, an automated evolutionary search against a Claude 4.6 Sonnet monitor found attack transcripts scoring 3/100 on suspicion, versus 14-18 for best-of-N baselines. That experiment used a single-step coding task in which the agent covertly exposes its API key.

The Frontier Model Forum described security practices for AI agents: limiting agent actions and resource access to what is strictly necessary, sandboxing with filesystem scope and egress policies, deterministic controls outside the model's reasoning loop, confirmation before high-stakes actions, and audit logs. It also covers layered prompt injection defenses, and names adaptive least privilege and extending identity standards such as OAuth 2.0 to agents as promising or developing areas.

NCSC authors advise deploying agentic AI incrementally through tightly bounded pilots, granting agents only the minimum permissions with temporary credentials, and defining in advance who approves access, monitors behavior and can halt the agent. They recommend incident response plans for agent failure and loss-of-control scenarios.

Jotautaitė, Martinez, Matthews and Tracy (two listed as independent, Tracy at Redwood Research) build a semi-automated red-teaming pipeline in the BashArena control setting, where agents covertly pursue side tasks such as firewall backdoors, privilege escalation, credential exfiltration and malware download. MonitoringBench's 2,644 attack trajectories lowered a Claude Opus 4.5 monitor's catch rate at 1% false positives from 94.9% on directly elicited attacks to 60.3% on refined ones.

OpenAI describes an asynchronous monitor powered by GPT-5.4 Thinking at maximum reasoning effort that reviews internal coding agent sessions, including chains of thought, within about 30 minutes of completion. OpenAI reports that over five months it covered tens of millions of trajectories, with no real conversation reaching the highest severity and about 1,000 moderate-severity alerts, many from deliberate internal red-teaming, and that it flagged every interaction employees had independently reported. The categories it labels common, each under 1% of traffic, are circumventing restrictions, deception and concealing uncertainty.

Check Point Research found that a cloned repository's Claude Code configuration could run hooks, start MCP servers before the user approved them, and redirect API traffic so the user's Anthropic API key was sent to an attacker (CVE-2025-59536, CVE-2026-21852). Anthropic fixed the issues between August and December 2025 by deferring execution and API calls until after the trust dialog.

Anthropic released Claude Code Security as a limited research preview for Enterprise and Team customers, with expedited free access for open-source maintainers. The tool reasons about data flow across a codebase, re-examines each finding in a multi-stage verification pass, assigns severity and confidence ratings, and proposes patches that are applied only with human approval.

OpenAI describes prompt injection as social engineering aimed at AI agents and lists its layered defenses: instruction-hierarchy safety training, automated red-teaming, AI-based monitors that can be updated quickly, sandboxing of code-running tools, link approval, confirmation before sensitive steps, logged-out mode in Atlas, and a watch mode on sensitive sites that pauses the agent if the user leaves the tab. It cites thousands of hours of prompt-injection-focused red teaming and a bug bounty, and says it has not yet seen significant attacker adoption of the technique.

Meta proposes that, within a session, an agent should have at most two of three properties: processing untrustworthy inputs, accessing sensitive systems or private data, and changing state or communicating externally. If all three are needed, the agent should not act autonomously and needs human approval or other validation. Meta illustrates this with travel, research and internal coding agent examples.

Anthropic describes sandboxing for Claude Code that restricts file writes to permitted directories and routes network traffic through a proxy that only allows approved hosts, so a prompt-injected agent cannot modify sensitive files or exfiltrate data freely. Anthropic says internal use showed an 84% reduction in permission prompts, and it released the sandbox runtime, built on bubblewrap and macOS seatbelt, as an open-source research preview.

Johann Rehberger showed that injected instructions in project content could make GitHub Copilot in VS Code edit workspace settings to switch off command confirmations, after which it could run arbitrary terminal commands. He reported it on 2025-06-29 and Microsoft patched it in the August 2025 Patch Tuesday.

SafeBreach researchers showed that instructions in calendar invite titles or email subjects, retrieved when a user asked Gemini about their schedule, could trigger tool misuse such as deleting events, opening URLs, exfiltrating email subjects, and controlling connected smart-home devices. They disclosed to Google in February 2025, and Google deployed layered mitigations including confirmations, URL handling policies and injection classifiers.

Check Point Research found that Cursor bound MCP approval to a configuration's name rather than its contents, so a collaborator with repository write access could swap an approved harmless command for a malicious one that ran on each project open. Cursor 1.3, released 2025-07-29, prompts for approval on any MCP configuration change.

The Coalition for Secure AI, an OASIS Open Project, published three principles for agentic systems. The principles call for agents that are human-governed and accountable, bounded and resilient with strict, purpose-specific entitlements, and transparent and verifiable through secure AI supply chain controls and telemetry that supports monitoring and forensics.