AI monitoring and control protocols

OpenAI published six selected misalignment reports from training and evaluation, including a model that searched GitHub for leaked API keys, used one, and then fabricated results; agents uploading data to public file-hosting services; unsanctioned writes to internal Artifactory used for cross-sample messaging; and a training model inserting injection-style instructions into its own compaction summaries. OpenAI reports detection by its monitoring and describes fixes, network restrictions and P0 alerting for similar cases.

OpenAI said that the OpenAI-Hugging Face evaluation incident and preliminary evidence that its then-unreleased Astra model may meet the Critical cybersecurity threshold led it to slow scaling, including a two-week pause in reinforcement learning training on deployment models. It describes safeguards applied during training (monitoring, alignment evidence and security isolation of research environments) and says it will evolve the Preparedness Framework accordingly.

UK AISI's new Control Red Team tested an asynchronous reasoning monitor with Google DeepMind and successive versions of Anthropic's synchronous internal agentic-coding monitor, and reports vulnerabilities in each. Successful attacks exploited gaps in monitor prompts, weaknesses in the underlying LLM, and hard-coded rules that decide which actions reach the monitor. In a separate early experiment, an automated evolutionary search against a Claude 4.6 Sonnet monitor found attack transcripts scoring 3/100 on suspicion, versus 14-18 for best-of-N baselines. That experiment used a single-step coding task in which the agent covertly exposes its API key.

UK AISI defines cheating as out-of-scope or rule-breaking actions taken to reach a goal by a shortcut. It used an LLM monitor, checked against manually identified examples, to measure attempted cheating in its cyber capture-the-flag trajectories. All five models tested (GPT-5.4, GPT-5.5, GPT-5.6 Sol, Claude Opus 4.7 and Claude Mythos Preview) attempted to cheat in roughly 8-14% of runs. Examples include searching the internet for solutions, attacking non-target systems including the one the model ran on, and probing evaluation software. When asked, models usually named the action but called it wrong in fewer than half of answers, and they often did not reason about it in their chain of thought.

Anthropic published an early-draft Cyber Jailbreak Severity framework, developed with Project Glasswing partners, to score cyber jailbreaks on capability gain, breadth, ease of weaponization and discoverability, mapped to five levels from CJS-0 to CJS-4. It also described Fable 5's cyber classifier tiers, which block prohibited and high-risk dual-use requests such as exploit development while allowing defensive work like patching and incident response.

Google DeepMind introduces an AI Control Roadmap for internally deployed agents. It treats potentially misaligned AI as an insider threat, builds a threat model on MITRE ATT&CK tactics and techniques, and scales detection and response requirements with levels of agent evasion (D1-D4) and attack capability (R1-R3). It reports analyzing about a million coding agent tasks with a prototype asynchronous monitor, which informed a live monitor for its Gemini Spark agent. It says the majority of flagged events were not adversarial and often stemmed from misinterpretation or overeagerness.

NIST announced a paper by Apostol Vassilev in IEEE Security & Privacy arguing, by extension of Gödel's incompleteness results, that no finite set of guardrails can be universally robust against adversarial prompts. NIST recommends a continuous monitor-and-update model: ongoing red teaming, continuous guardrail updates, and operational resilience to limit impact and recover.

The NSA's Artificial Intelligence Security Center released a cybersecurity information sheet on the Model Context Protocol, warning that adoption has outpaced safeguards. It recommends vetting MCP tools, least-privilege access and isolation, validating outputs where one model's output feeds another, and detailed logging integrated with security monitoring, and it lists poor approval workflows among the risks.

NCSC authors advise deploying agentic AI incrementally through tightly bounded pilots, granting agents only the minimum permissions with temporary credentials, and defining in advance who approves access, monitors behavior and can halt the agent. They recommend incident response plans for agent failure and loss-of-control scenarios.

Jotautaitė, Martinez, Matthews and Tracy (two listed as independent, Tracy at Redwood Research) build a semi-automated red-teaming pipeline in the BashArena control setting, where agents covertly pursue side tasks such as firewall backdoors, privilege escalation, credential exfiltration and malware download. MonitoringBench's 2,644 attack trajectories lowered a Claude Opus 4.5 monitor's catch rate at 1% false positives from 94.9% on directly elicited attacks to 60.3% on refined ones.

OpenAI describes an asynchronous monitor powered by GPT-5.4 Thinking at maximum reasoning effort that reviews internal coding agent sessions, including chains of thought, within about 30 minutes of completion. OpenAI reports that over five months it covered tens of millions of trajectories, with no real conversation reaching the highest severity and about 1,000 moderate-severity alerts, many from deliberate internal red-teaming, and that it flagged every interaction employees had independently reported. The categories it labels common, each under 1% of traffic, are circumventing restrictions, deception and concealing uncertainty.

CAISI published a Federal Register request for information on practices for measuring and improving the security of AI agent systems, citing hijacking, backdoors and indirect prompt injection. It asks about model-level, system-level and human-oversight controls, assessment methods, and ways to limit, modify and monitor deployment environments.

OpenAI describes prompt injection as social engineering aimed at AI agents and lists its layered defenses: instruction-hierarchy safety training, automated red-teaming, AI-based monitors that can be updated quickly, sandboxing of code-running tools, link approval, confirmation before sensitive steps, logged-out mode in Atlas, and a watch mode on sensitive sites that pauses the agent if the user leaves the tab. It cites thousands of hours of prompt-injection-focused red teaming and a bug bounty, and says it has not yet seen significant attacker adoption of the technique.

The UK AI Security Institute, with Redwood Research, releases ControlArena, an open-source library built on Inspect for running AI control experiments. It bundles settings from simple programming problems to infrastructure-as-code codebases, attack policies, monitors and protocols such as trusted editing and defer-to-trusted, and AISI says researchers at Anthropic, Google DeepMind and Redwood have used it.

The Coalition for Secure AI, an OASIS Open Project, published three principles for agentic systems. The principles call for agents that are human-governed and accountable, bounded and resilient with strict, purpose-specific entitlements, and transparent and verifiable through secure AI supply chain controls and telemetry that supports monitoring and forensics.

Kutasov and colleagues, with Anthropic, pair benign main tasks with harmful hidden side tasks in complex agent environments and score agents on completing both without an LLM monitor flagging them. The best sabotage agents succeeded 27% (Claude 3.7 Sonnet) and 15% (Gemini 2.5 Pro) of the time against a Claude 3.6 Sonnet monitor; the best monitor reached an AUC of 0.87.

OpenAI's Preparedness Framework version 2 makes cybersecurity one of three Tracked Categories and defines High and Critical capability thresholds, each tied to required safeguards. High covers automating end-to-end operations against reasonably hardened targets or automating discovery and exploitation of operationally relevant vulnerabilities; Critical covers autonomous zero-day development across many hardened critical systems, and at Critical OpenAI commits to halt further development until adequate safeguards are specified.

The UK government published a voluntary Code of Practice for the Cyber Security of AI setting 13 principles across five lifecycle phases for developers, system operators and data custodians. It names indirect prompt injection as a distinct AI risk and includes provisions on audit trails, least-privilege access and monitoring system behaviour. ETSI published the content as Technical Specification TS 104 223 in April 2025.