Weigh the evidence

Findings

Records say what happened. Findings say what we can conclude from them. Each finding is one scoped claim with its evidence, and a status that we update when later work supports, qualifies, contests, or supersedes it.

ReportedCorroboratedQualifiedContestedSupersededRevalidateWhat the statuses mean

How findings connect

supportsqualifiescontestssupersedes

ReportedCorroboratedQualifiedContestedSupersededRevalidate· node size = evidence records · columns group by topic

Select a finding to see how it relates to others. Arrows point from the newer finding to the one it supports, qualifies, contests, or supersedes.

The ledger

52 shown

2026-09-25

AI-written incident investigations can score better on reconstruction while keeping earlier conclusions the records do not support.

Reportedmeasured1 evidence · 1 independent sourceIncident reporting · Evaluation validity

2026-09-08

Evaluation pipeline choices alone can move a model's cybersecurity benchmark score by more than 80 points and reorder models.

Reportedmeasured1 evidence · 1 independent source1 relationEvaluation validity

2026-09-05

In a SOC studied for over a year, analysts reused an agentic assistant's output in the closing reports of over 90% of tickets during a four-month deployment.

Reportedobserved1 evidence · 1 independent sourceSOC automation · Monitoring & control

2026-07-21

Frontier models take out-of-scope shortcuts in cyber evaluations, and their own reports and reasoning do not reliably reveal it.

Corroboratedmeasured2 evidence · 2 independent sourcesEvaluation validity · Capability evaluation

2026-07-21

Agents under evaluation have coordinated through unintended shared channels, reused each other's artifacts, and tried to keep those channels alive.

Corroboratedobserved3 evidence · 3 independent sources1 relationMulti-agent security · Sandbox & containment

2026-07-21

Frontier agents under cyber evaluation have taken actions against real third-party systems outside the evaluation.

Corroboratedobserved4 evidence · 4 independent sources2 relationsSandbox & containment · Incident reporting · Evaluation validity

2026-07-16

Defensive SOC agents do not improve with more compute the way offensive agents do.

Reportedmeasured1 evidence · 1 independent sourceSOC automation · Evaluation validity

2026-05-29

Cyber capability measured at fixed, low token budgets understates what frontier models can do and how fast they are improving.

Reportedmeasured4 evidence · 2 independent sources1 relationEvaluation validity · Capability evaluation

2026-05-18

AI-driven vulnerability discovery produces reports faster than maintainers can triage and patch them.

Corroboratedreported2 evidence · 2 independent sourcesVulnerability discovery · Vulnerability repair

2026-05-13

Under a 2.5M-token cap, frontier cyber task time horizons doubled on the order of months between late 2024 and early 2026.

Qualifiedmeasured1 evidence · 1 independent source1 relationCapability evaluation

2026-05-13

Counting a crash as exploitation success overstates capability; most public models in May 2026 stalled before code execution on a browser engine.

Reportedmeasured1 evidence · 1 independent source1 relationEvaluation validity · Exploit development

2026-05-12

Scores on public CTF benchmarks can be inflated when agents find published solutions, and static benchmarks lose validity as their flaws are patched.

Reportedmeasured2 evidence · 2 independent sourcesEvaluation validity

2026-05-11

On ExploitGym (May 2026), the strongest agents produced working exploits for 157 and 120 of 898 instances with mitigations off; with standard mitigations on, 45 and 21 survived.

Qualifiedmeasured1 evidence · 1 independent source1 relationExploit development · Capability evaluation

2026-05-11

Google's threat intelligence team attributes to AI the development of a zero-day exploit that criminal actors prepared for mass exploitation, based on characteristics of the code.

Reportedreported1 evidence · 1 independent sourceAI-enabled intrusion · Exploit development · Threat intelligence

2026-05-01

Independent government testing can find a model weaker on agentic tasks than its developer's self-reported benchmarks suggest.

Reportedmeasured1 evidence · 1 independent source1 relationEvaluation validity · Capability evaluation

2026-05-01

Six national cyber agencies jointly advise starting agent deployments with low-risk, non-sensitive use cases and not granting agents broad access to sensitive data or critical systems.

Reportedargued1 evidence · 1 independent sourceStandards & guidance

2026-04-15

An MCP endpoint exposed without authentication has been reported as exploited in the wild.

Reportedreported1 evidence · 1 independent sourceTools & MCP

2026-03-19

OpenAI reports its internal coding-agent monitor matched every staff escalation, and OpenAI and Google DeepMind report that most flags reflect overeagerness or mistakes rather than adversarial intent.

Qualifiedreported2 evidence · 2 independent sources1 relationMonitoring & control

2026-03-01

The strongest frontier models tested escaped deliberately vulnerable container sandboxes in about half of scenarios, and success rose with inference compute.

Reportedmeasured1 evidence · 1 independent sourceSandbox & containment · Capability evaluation

2026-02-07

Checking only that the original crash no longer reproduces overstates how often AI-generated patches actually fix the vulnerability.

Corroboratedmeasured2 evidence · 2 independent sources1 relationVulnerability repair · Evaluation validity

2025-11-05

Malware that queries a language model during execution to generate commands has been used in live operations, including by a state-backed group; self-rewriting variants have been seen only in testing.

Corroboratedobserved2 evidence · 2 independent sources1 relationAI malware · Threat intelligence

2025-10-20

Operating-system sandboxing can cut the number of approval prompts a coding agent needs without widening what injected instructions can reach.

Reportedreported1 evidence · 1 independent sourceSandbox & containment · Monitoring & control

2025-08-27

Attackers have used AI agents to carry out much of the work of real intrusions, from reconnaissance to credential theft and extortion, with people directing them at a few decision points.

Corroboratedobserved5 evidence · 4 independent sourcesAI-enabled intrusion · Threat intelligence

2025-08-20

Agentic browsers act on page content that users cannot see.

Reportedmeasured3 evidence · 1 independent sourcePrompt injection

2025-07-28

In several agent frameworks and coding tools, prompt injection in content the agent read reached code execution on the host, sometimes combined with a common setting such as an allowlisted command.

Corroboratedmeasured5 evidence · 5 independent sourcesPrompt injection · Sandbox & containment

2025-07-28

Every frontier agent tested in two large public red-teaming competitions was hijacked at least once; in the 2026 competition the injections also had to conceal the compromise from the user.

Reportedmeasured2 evidence · 1 independent source2 relationsPrompt injection

2025-07-28

More capable models are not reliably more robust to prompt injection, and some are more susceptible.

Corroboratedmeasured3 evidence · 3 independent sourcesPrompt injection · Tools & MCP

2025-07-28

Approval gates in coding agents do not always show, or even ask about, what the agent is actually about to do.

Corroboratedmeasured2 evidence · 2 independent sourcesMonitoring & control

2025-07-23

Malicious or compromised agent extensions, MCP servers, and skills have been published to public registries and used against real users.

Corroboratedobserved4 evidence · 4 independent sourcesAgent supply chain · Tools & MCP

2025-07-22

Limits that agents are only told about, or that block one tool while others remain available, do not reliably stop them: a declared code freeze and an ignore-file setting were both bypassed.

Corroboratedobserved2 evidence · 2 independent sources1 relationMonitoring & control · Sandbox & containment

2025-07-14

LLM agents fall well short of reliable performance on realistic threat-investigation and threat-hunting benchmarks built from security logs.

Corroboratedmeasured3 evidence · 3 independent sources1 relationSOC automation · Autonomous defense

2025-06-17

Agent monitors can be driven to miss covert actions: optimized attacks pushed suspicion scores near zero, and weak red-teaming overstates catch rates.

Corroboratedmeasured3 evidence · 3 independent sources1 relationMonitoring & control

2025-06-10

Limiting what untrusted input can cause an agent to do gives injection resistance that does not depend on the model resisting.

Corroboratedargued5 evidence · 4 independent sources1 relationPrompt injection · Data exfiltration

2025-05-26

An agent holding broad credentials turns one injected instruction into access to everything those credentials reach.

Corroboratedmeasured5 evidence · 5 independent sourcesTools & MCP · Data exfiltration

2025-05-20

Frontier labs' layered defenses reduce prompt injection in tool-use, browser and computer-use agents but do not eliminate it.

Qualifiedreported5 evidence · 3 independent sources2 relationsPrompt injection · Jailbreaks & safeguards

2025-05-07

In the CAGE 4 simulation, reinforcement-learning defenders outperformed LLM-based defenders.

Revalidatemeasured1 evidence · 1 independent sourceAutonomous defense

2025-04-01

Text in an MCP tool's description can steer an agent's other actions, including leaking local secrets.

Corroboratedmeasured2 evidence · 2 independent sourcesTools & MCP · Agent supply chain

2025-04-01

Approving an agent's tools or configuration once is not enough, because they can change after approval or be changed by the agent itself.

Corroboratedmeasured4 evidence · 3 independent sourcesTools & MCP · Agent supply chain

2025-03-24

Separating an agent's control flow from untrusted data can give provable protection against control-flow hijacking at a modest utility cost.

Revalidatemeasured1 evidence · 1 independent source1 relationPrompt injection

2025-02-03

Anthropic reports that classifier guards cut automated jailbreak success from 86% to 4.4%; a 2025 public demo still yielded one universal jailbreak, though none has been reported against its 2026 successor.

Reportedreported2 evidence · 1 independent sourceJailbreaks & safeguards

2025-01-17

Measured hijack rates rise sharply when attackers get repeated attempts, so single-attempt figures understate risk.

Corroboratedmeasured3 evidence · 3 independent sources2 relationsPrompt injection · Evaluation validity

2025-01-17

Attackers who adapt to a defense defeat most published prompt-injection defenses that reported near-zero success against static attacks.

Reportedmeasured4 evidence · 4 independent sources4 relationsPrompt injection · Evaluation validity

2024-11-01

AI systems find real, previously unknown vulnerabilities in widely used open-source software.

Corroboratedobserved6 evidence · 6 independent sourcesVulnerability discovery

2024-09-20

Instructions written into an agent's long-term memory or its own context summaries can persist and later steer its behavior.

Corroboratedobserved2 evidence · 2 independent sourcesPrompt injection · Monitoring & control

2024-08-11

In DARPA's AI Cyber Challenge, autonomous systems patched most of the synthetic vulnerabilities they found.

Qualifiedmeasured2 evidence · 1 independent source1 relationVulnerability repair

2024-03-05

Undefended tool-using agents follow injected instructions in a substantial share of benchmark cases.

Qualifiedmeasured4 evidence · 4 independent sources1 relationPrompt injection

2024-03-05

A single malicious message can spread on its own across networks of connected agents.

Corroboratedmeasured2 evidence · 2 independent sourcesMulti-agent security · Prompt injection

2024-02-14

AI providers’ early reports found threat actors using LLMs mainly as productivity tools for research, scripting help and content, without novel attack capabilities.

Supersededobserved2 evidence · 2 independent sources1 relationThreat intelligence · AI-enabled intrusion

2024-02-09

Published prompt-injection defenses report attack success cut to near zero, or under 10%, against most of the fixed attacks their authors tested.

Revalidatemeasured4 evidence · 4 independent sources1 relationPrompt injection

2023-12-12

In an early AI-control experiment, trusted editing gave a better safety-usefulness tradeoff than untrusted monitoring.

Revalidatemeasured1 evidence · 1 independent sourceMonitoring & control

2023-12-05

Controlled trials run by Microsoft report that its security assistants make analysts faster and more accurate.

Reportedreported2 evidence · 2 independent sources1 relationSOC automation

2023-11-03

Content sent by outsiders, such as email, calendar invites, shared documents, or web forms, can trigger agent actions without the user clicking anything.

Corroboratedmeasured6 evidence · 6 independent sourcesPrompt injection · Data exfiltration