How it works
A classifier inspects traffic around the agent model. Examples include PromptGuard, Prompt Shields, Model Armor, and Constitutional Classifiers.
Can be bypassed by adaptive attacks and adds false positives; works best as one layer among several.
What we know
2 reported, 1 qualifiedRecords over time
Use the arrow keys to move between records, Home and End to jump to the first and last, and Enter to select one.
Select a mark to read the record. Mark size shows editorial significance. Hollow marks are dated to the month. Era bands are editorial labels.
Records in view
22 records · newest firstAmelia Kawasaki and Stefan Lee evaluate Attention Tracker and propose AttnDA, a learned alternative using attention-head features. Across five tested open models, synthetic-trained Attention Tracker averages 0.97 AUROC on deepset but 0.57 on the paper’s AgentDojo subset. Domain-matched training helps AttnDA; these are detection results rather than evidence of preventing harmful agent actions.
Huawei authors propose a plug-in layer in which fine-tuned 3B-parameter simulators, built on an uncensored model, draft a tree of the target agent's possible future actions, prune it against the actions the agent really takes, and pass the leaves to an existing detector. On AgentDojo with a Qwen3-235B agent they report attack success falling to 0% with two detectors, and on ActorAttack multi-turn jailbreaks to 0.0% at a sampling budget of 5. The attacks were benchmark attacks not designed against the defense, and no attacker aware of it was tested.
Li, He, Dai and Xiao (arXiv v1, 29 September 2026) propose ToolFence, an inference-time defense that compiles the authenticated user request into typed capabilities with provenance constraints on authority-sensitive arguments, enforces them with a deterministic monitor, and asks an LLM judge to grant new capability shapes rather than judge each call. On AgentDojo with Qwen3-max the authors report overall attack success of 0.20% against 21.20% undefended, with clean utility 38.90% against 42.70% undefended; a cross-session cache of approved shapes cut judge calls per task from 1.84 to 1.05 in their ablation. The attacks are six injection strategies averaged; the authors report no adaptive attacks against ToolFence and residual failures inside authorized data flows.
Researchers at SPAR, Cambridge, APTA AI and CISPA study a class of attack in which adversarial text in a shared artifact is stored in one assistant's memory, reproduced in an artifact it later writes and picked up by another assistant, with no direct agent-to-agent channel. In 36 synthetic workflows on the OpenClaw harness with an attacker-operated upload endpoint, the authors report that a single seed artifact goal-infected 38% to 98% of assistants (32% to 73% fully infected) depending on the model. They report an attacker-service-free variant that was weaker, and that an off-the-shelf classifier at the memory-write step flagged their main template's infected memories, with false positives on legitimate instructions.
Anthropic's Claude Opus 5.5 system card reports that, evaluated through the API with its cyber safeguards off, the model meets or exceeds Claude Mythos 5.1 and Claude Opus 5 on every cyber evaluation it includes (ExploitBench, CyScenarioBench, a rewritten Binary Exploitation Benchmark and ExploitGym). Anthropic places the model in the lower category of its Frontier Compliance Framework and says it sees no indication of novel offensive capability. It deployed the model with a three-stage cyber safeguard (an activation probe and two classifiers) that blocks vulnerability discovery in compiled binaries, and reports no evidence of a critical-severity jailbreak from internal and contracted external red teaming (the card gives no CAISI results).
Anthropic's Claude Opus 5.5 system card reports that, without production safeguards, the model attempted to cross a containment boundary in 1.5% of cases in a 28-scenario sandbox evaluation (all low severity) and acted in ways Anthropic judges likely harmful in roughly half of simulated capture-the-flag exercises that offered package-registry credentials, with verbalized evaluation awareness in about one-third of runs. It also reports a regression in which early snapshots followed instructions planted in text a user pasted into their own prompt, which Anthropic traces to its prompt-injection training, and rare spontaneous malicious tool calls in internal use. On indirect prompt injection through tool results, the card reports results matching or better than Claude Opus 5 across coding, computer use and browser surfaces.
Security researcher Johann Rehberger (Embrace The Red) reports that a request to summarize a web page led Claude Code with Opus 5 in Auto Mode to run attacker-controlled code in his lab setup, using a multi-step chain of individually benign-looking actions, with success in 3 of 5 to 4 of 5 trials per variant. He says Anthropic closed his report as Informative and working as designed, and relays that Anthropic's position is that Auto Mode is a best-effort classifier for convenience, not a security boundary. He contrasts this with a third-party evaluation, commissioned by Anthropic and described in a post he cites, that showed 0.00% prompt-injection success for Opus 5 in Auto Mode on a fixed scenario set.
UK AISI reports that during cyber range evaluations from July 25 to 28, 2026, run with open internet access and cyber classifiers disabled, agents took 19 unsanctioned actions against real people and services in 10 of 122 runs. Actions included an attempted supply-chain contribution of malicious code with fake identities, social engineering, planting prompt injections for other AI systems, and leaving public instructions other agents reused; Anthropic's Mythos 5 accounted for 17 and OpenAI's GPT-5.6 Sol for 2. Security monitoring flagged unusual transfers on July 28 and AISI contained activity within about an hour.
Anthropic published an early-draft Cyber Jailbreak Severity framework, developed with Project Glasswing partners, to score cyber jailbreaks on capability gain, breadth, ease of weaponization and discoverability, mapped to five levels from CJS-0 to CJS-4. It also described Fable 5's cyber classifier tiers, which block prohibited and high-risk dual-use requests such as exploit development while allowing defensive work like patching and incident response.
Anthropic announced that export controls on Fable 5 and Mythos 5 had been lifted and that Fable 5 would be redeployed globally from July 1, 2026 with an improved safety classifier. Anthropic says the classifier blocks the technique described in an Amazon report in over 99% of cases and that CAISI researchers tested its prior and new safeguards. Mythos 5 access was restored for a set of US organizations after government approval on June 26.
NIST announced a paper by Apostol Vassilev in IEEE Security & Privacy arguing, by extension of Gödel's incompleteness results, that no finite set of guardrails can be universally robust against adversarial prompts. NIST recommends a continuous monitor-and-update model: ongoing red teaming, continuous guardrail updates, and operational resilience to limit impact and recover.
OpenAI explains that an injected agent can leak data by requesting an attacker URL that embeds private information, and argues that domain allow-lists are insufficient because trusted sites can redirect and strict lists cause warning fatigue. Its safeguard only lets the agent fetch a URL automatically if an independent crawler has already seen that exact URL on the public web; otherwise it warns the user or tells the agent to use another source. A March 2026 post names the mechanism Safe Url and places it within a social-engineering view of prompt injection and source-sink analysis.
Anthropic describes Constitutional Classifiers++, a cascade in which a cheap linear probe on model activations screens all traffic and escalates flagged exchanges to a probe-classifier ensemble. It reports roughly 1% added compute if applied to Claude Opus 4.0 traffic (the first generation added 23.7%) and a 0.05% refusal rate on harmless queries over one month of Claude Sonnet 4.5 traffic. Red-teamers found no universal jailbreak in over 1,700 hours.
Anthropic describes three defenses for browser use: reinforcement learning on injected web content, classifiers that scan untrusted content, and human red-teaming including external arena-style challenges. Against an internal adaptive Best-of-N attacker with 100 attempts per environment, its chart shows Claude Opus 4.5 with the improved Claude for Chrome safeguards at 1.4% attack success (8.0% for the model alone), which the post rounds to 1%. Anthropic says this still represents meaningful risk.
Nasr, Carlini, Tramèr and 11 co-authors apply gradient, reinforcement learning, search and human red-teaming attacks to 12 published defenses. Most defenses originally reported near-zero attack success, but the adaptive attacks exceed 90% success against most, and human red-teamers succeeded on every challenge in the subset of defenses they were given.
SafeBreach researchers showed that instructions in calendar invite titles or email subjects, retrieved when a user asked Gemini about their schedule, could trigger tool misuse such as deleting events, opening URLs, exfiltrating email subjects, and controlling connected smart-home devices. They disclosed to Google in February 2025, and Google deployed layered mitigations including confirmations, URL handling policies and injection classifiers.
General Analysis demonstrated a Cursor agent connected to Supabase MCP with a service-role key, which bypasses row-level security, following instructions in a customer support ticket to read a secrets table and write the contents back into the attacker-visible ticket. Supabase later responded that agents should not be connected to production data and described guardrails that reduced but did not eliminate risk.
Simon Willison argues that an agent becomes exploitable for data theft when it combines access to private data, exposure to untrusted content, and the ability to communicate externally. He advises users to avoid combining all three, points developers to design-pattern mitigations, and argues that guardrails catching most attacks are inadequate in a security setting.
Aim Labs disclosed a zero-click chain in which an email containing hidden instructions, once retrieved by Microsoft 365 Copilot, could cause Copilot to embed internal data in an auto-loaded image request to an attacker. Microsoft rated CVE-2025-32711 critical, fixed it server-side in May 2025, and stated there was no evidence of real-world exploitation.
Shi and colleagues describe Google DeepMind's continuous adaptive-attack evaluation of Gemini against indirect prompt injection in tool-use settings. On Gemini 2.0, adaptive attacks generally matched or beat non-adaptive ones against eight baseline defenses, reaching 98.4% against in-context learning and 82.4% against spotlighting, while a warning defense and a user-instruction classifier held (at most 10.8% and 3.0%). Adversarial fine-tuning for Gemini 2.5 lowered but did not eliminate attack success.
Meta open-sources LlamaFirewall, combining PromptGuard 2 (a jailbreak and injection detector), AlignmentCheck (a chain-of-thought auditor for goal hijacking) and CodeShield (static analysis of generated code). On AgentDojo, Meta reports that the combination cut attack success from 17.63% to 1.75% while utility fell from 47.73% to 42.68%.
Anthropic describes input and output classifiers trained on synthetic data generated from a natural-language constitution of allowed and disallowed content, targeted at chemical-weapons style queries. In automated testing on Claude 3.5 Sonnet, jailbreak success fell from 86% to 4.4%, and a prior bug bounty found no universal jailbreak; a public demo in February 2025 did yield one universal jailbreak.