Topics/Defense

Autonomous defense

Agents that detect, contain, and respond without waiting for a person.

33 records2 findings2 openings7 benchmarks and toolsLatest record
Start here

Defenders' agents

What AI agents can and cannot yet do for security operations, from assisted analysts to autonomous response in simulation.

  1. Microsoft randomized controlled trial measures Security Copilot effect on analyst speed and accuracy
    An early controlled trial of an analyst assistant.
  2. TTCP releases CAGE Challenge 4, a multi-agent autonomous cyber defence environment
    A shared simulation for autonomous defense.
  3. UC Santa Cruz study integrates LLM agents into CAGE 4 and finds RL defenders still outperform them
    LLM defenders compared with reinforcement learning.
  4. Baseline agent approaches can fall well short of reliable investigation and triage on security-log benchmarks.
    How agents fare on realistic investigation benchmarks.
  5. Year-long SOC fieldwork finds analysts reused an agentic AI companion's output in over 90% of tickets
    What happens in a real SOC.
  6. Why don't defensive agents get better with more compute?
    The open question.
RangeLanes
27 of 33 records in view

Use the arrow keys to move between records, Home and End to jump to the first and last, and Enter to select one.

Agents find real bugsAgents in real operationsGated capability, incidents in the labAttackCapabilityDefensePolicyJan 25Jul 25Jan 26Jul 26
Full record · drag to choose a range
202420252026

Select a mark to read the record. Mark size shows editorial significance. Hollow marks are dated to the month. Era bands are editorial labels.

Records in view

27 records · newest first
Nov 2026
Nov 10, 2026
AlphaProof Code checks specified Signal security properties at machine-code level
DefensePaperGoogle DeepMind, Google

Google DeepMind describes AlphaProof Code, combining human-approved specifications with agent-generated proofs checked by Lean. Its libsignal study proves specified properties of PQXDH and the Triple Ratchet at the AArch64 machine-code level. Assurance depends on the specifications, formal semantics and other stated trust assumptions; it does not cover every property of the Signal application.

Nov 2, 2026
Trail of Bits reports Codex-assisted backports delivered for 149 Linux CVEs
DefenseEvaluation reportTrail of Bits, OpenAI

Trail of Bits reports that its Codex workflow helped deliver fixes for 149 distinct CVEs into stable Linux kernel releases by October 29, with additional package work for Fedora and Ubuntu. Agents prepare backports and evidence; engineers review and submit them to maintainers. Reported human-time savings exclude setup, regression testing and maintainer acceptance.

Oct 2026
Oct 29, 2026
OpenAI describes expanded agent monitoring, with workload-specific coverage and response limits
DefenseEvaluation reportOpenAI

OpenAI describes expanded misalignment monitoring across training, evaluations, internal use and selected external deployments, using activation probes, model reviewers, investigative agents and telemetry. Within its internal system, highest-priority alerts could page responders at any hour for frontier reinforcement learning; most other traffic had business-hours paging, and internal Codex paging was disabled.

Oct 21, 2026
Trail of Bits releases patching and regression-review plugins after finding 21 regressions in merged patches
DefenseTool releaseTrail of Bits, OpenAI

Trail of Bits releases patching-bugs and regression-hunt, plugins developed through its Patch the Planet work with OpenAI. It reports 21 confirmed regressions among more than 130 merged pull requests and preliminary testing in which regression-hunt produced 25% fewer false-positive findings than a baseline Codex review. The comparison used those 21 regressions, and stricter evidence requirements also left some known regressions unconfirmed; patching-bugs itself had not yet received the planned evaluation.

Oct 19, 2026
Microsoft extends Project Perception to treat AI agents as insiders, with agent identities, task-scoped permissions and risk-based access
DefenseTool releaseMicrosoft

Microsoft announced on 2026-10-19 that it is extending Project Perception to govern AI agents as potential insiders: discovering local agents such as Claude Code, OpenAI Codex and GitHub Copilot, letting discovered agents receive their own Entra identity, scoping permissions to a task and its duration, and pairing a continuously updated agent risk assessment with conditional access that can narrow access, require human approval, or isolate and stop an agent. Microsoft frames the design around keeping security controls outside the model, so that a manipulated or compromised agent cannot bypass them. The post is a product announcement and gives no availability dates or measurements of effectiveness.

Oct 13, 2026
CyberArena benchmark pits 12 frontier agents against each other in live attack-and-defense matches over 466 games
CapabilityBenchmarkUC Berkeley, UC Santa Cruz, Purdue University

A team from 2077AI and several universities presents CyberArena, in which two model-harness entrants host identical vulnerable services in isolated sandboxes and, over three 20-minute rounds, try to capture rotating flags from the opponent while patching their own service under health checks. They report standings for 12 agents over 466 valid games on 40 challenges, with GPT-5.6 Sol at an expected score of 78.6 and Claude Opus 5 at 73.8, and say contested play lowers capture rates relative to solo play. An LLM-run audit of all trajectories reports recurring defender errors such as partial patches and repair claims resting on stale evidence; the authors note it is not human verification of every finding.

Oct 12, 2026
Collinear's CWE-bench: 120 private audit-and-repair tasks, with three of eleven agents tied at 68% pass@1
DefenseBenchmarkCollinear

Collinear researchers introduce CWE-bench, a private benchmark of 120 repository audit-and-repair tasks built on real open-source repositories with injected vulnerabilities across 73 CWE types, eight languages and all ten OWASP Top 10:2025 categories; agents receive a threat model but not the weakness, its location or the number of vulnerable sites. A patch passes only if a programmatic verifier finds the exploits blocked and existing functionality tests passing, and the authors check the verifier against a three-judge agentic panel. They report that Grok 4.7, Gemini 4 Argon and GPT-6 Astra tie at 68% pass@1, that Claude Opus 5.5 reaches 67%, and that pass@4 for the tied agents ranges from 74% to 81%; the results are author-run on a task set outsiders cannot inspect.

Oct 8, 2026
Anthropic launches critical-infrastructure support and free open-source scanning
DefenseProgramAnthropic

Anthropic launches its Cyber Mission with a Critical Infrastructure Defense Program and an opt-in OSS Scanner offering free periodic security scans to open-source maintainers. Scanner reports are model-generated and sent without human review; Anthropic continues human-verified disclosure for projects unable to absorb that volume. The announcement describes efforts and expected performance rather than an independently measured reduction in security risk.

Oct 7, 2026
ALERT-BENCH finds triage failures; AIDA reduces missed attacks in telemetry replay
DefenseBenchmarkVirginia Tech, North Carolina State University, Google

Researchers introduce ALERT-BENCH, requiring agents to investigate detector-generated alerts through a live SIEM over replayed public telemetry. On the main 1,247-alert sample, five reasoning approaches each miss at least 40.4% of attack-related alerts, while the authors’ AIDA architecture reduces the false-negative rate to 3.1% under the same GPT-4.1 configuration, with more alerts escalated to analysts. Additional scenario and model tests remain within the studied dataset family; analyst use and independent SOC validation are untested.

Sep 2026
Sep 30, 2026
APTInvestBench finds autonomous investigators lose citation support when telemetry changes
DefenseBenchmarkZhongguancun Laboratory

Zhongguancun Laboratory researchers introduce APTInvestBench, built from report-informed attack reconstructions under varied log-collection conditions. In their ten-scenario comparison of eleven models, agents acquire sufficient evidence for 44.3% of recoverable attack actions on average, but their formal citations support 25.0%; similar aggregate citation coverage can conceal losses of support for actions that remain recoverable. These are controlled benchmark investigations, not measurements of deployed SOC performance.

Sep 28, 2026
Artificial Analysis launches a Cyber Index and industry alliance scoring models on source-level vulnerability finding and fixing
DefenseBenchmarkArtificial Analysis, Collinear, IBM

Artificial Analysis announced a Cyber Index Alliance, with Collinear AI, IBM, NVIDIA and Vercel as launch partners, and a v1 Cyber Index that combines three source-code evaluations: repository audit and patch (CWE-Bench-AA, 120 private tasks), finding expert-verified vulnerabilities (DeepsecBench-AA) and discovering, reproducing and patching memory-safety bugs (CyberGym-E2E-AA, 131 tasks). Artificial Analysis says it excludes exploit building, runs all three on its open-source Stirrup harness, and reports safety refusals separately from scores. The article reports failure-mode statistics and says several frontier models refuse almost every CyberGym-E2E-AA task; per-model index scores appear only in charts.

Sep 26, 2026
CyberClear benchmarks LLM agents on reconstructing APT attack chains from long defender logs
DefenseBenchmarkHong Kong Polytechnic University, OpenClaw, Southeast University

Chen and colleagues (arXiv v1, under review for ICLR 2027) introduce CyberClear, 450 instances built from public APT log datasets in which an agent must turn long defender logs, without prior attack clues, into a provenance graph with ATT&CK-mapped steps. References were generated by an LLM and passed automatic verification and expert review; scoring compares graph code with five LLM-judge dimensions. They also propose CyberProvenance, a multi-agent harness that reproduces predicted attack steps in isolated lab environments and refines the graph, and report it best on most semantic metrics while the highest Strict score remains 0.6477 out of 1.

Sep 24, 2026
Google's PageBreak agent finds over 500 XSS bugs in its own web apps using deterministic validators
DefenseTool releaseGoogle

Google's Product Security team describes PageBreak, an internal agent mostly using Gemini models that hunts vulnerabilities in Google's first-party web applications and only reports findings confirmed by non-AI validators against running applications. Google reports over 500 XSS vulnerabilities found with near-zero false positives, while apps on its high-assurance web frameworks yielded only 2 XSS bugs as of 4 September 2026.

Sep 21, 2026
UK NCSC blog argues defenders cannot use agentic AI as attackers do and proposes a five-dimension action risk framework
PolicyGuidanceUK National Cyber Security Centre, CETaS (Alan Turing Institute)

Dave Chismon, the NCSC's CTO for Architecture, argues that offensive problems are mostly technical with clear success states while defensive problems are mostly organizational, so defenders cannot put AI to work as attackers can and agentic defense may struggle to keep pace with AI-enabled attacks. He proposes rating a defensive action's riskiness on five dimensions (potency, scope, criticality, rollout confidence, recoverability) and starting with low-potency tasks where AI advises a human. The post also says the NCSC and DCMS are working on a national-scale agentic cyber defence ecosystem called Cyber Shield and that the NCSC seeks research on proving low-risk actions are low risk.

Mar 2026
Jan 2026
Jan 8, 2026
PNNL uses a Claude-based agent to speed adversary emulation against a water treatment plant model
DefensePaperAnthropic, Pacific Northwest National Laboratory, CISA

Anthropic reports that Pacific Northwest National Laboratory built a scaffold around Claude Sonnet 4 to automate adversary emulation against a high-fidelity cyber-physical model of a water treatment plant used for CISA. PNNL estimates attack reconstruction took three hours instead of multiple weeks; in one run the model switched to a different known privilege-escalation technique when a provided tool failed.

Dec 2025
Dec 16, 2025
NIST releases preliminary draft Cyber AI Profile (IR 8596) under CSF 2.0
PolicyFrameworkNIST

NIST published a preliminary draft Cybersecurity Framework Profile for Artificial Intelligence, aligned with CSF 2.0. It is organized around three focus areas: securing AI systems, using AI for cyber defense, and thwarting AI-enabled cyberattacks, with comments due January 30, 2026.

Dec 3, 2025
CISA, ASD and partners issue principles for securely integrating AI, including agents, into OT
PolicyGuidanceCISA, Australian Signals Directorate (ACSC), NSA Artificial Intelligence Security Center

CISA and the Australian Signals Directorate, with NSA, FBI and national cyber agencies of Canada, Germany, the Netherlands, New Zealand and the UK, published four principles for integrating AI into operational technology. The guidance explicitly covers machine learning, LLM-based AI and AI agents because of the security and safety challenges they pose in industrial environments.

Nov 2025
Nov 17, 2025
Microsoft RCT finds phishing triage agent raised analysts' true positives per minute up to 6.5x
DefensePaperMicrosoft

Microsoft reports a randomized controlled trial of its own Security Copilot Phishing Triage Agent. In the trial, 167 external security analysts each triaged a 25-email queue drawn from a curated corpus of emails reported by Microsoft employees. In the scenario where the agent classified every corpus email correctly, analysts with the agent found 6.5 times as many true positives per minute as the control group and scored 77% higher on F1; with the agent's accuracy set to 80% and a 20% malicious rate, the productivity gain fell to 3.1 times. Analysts with the agent spent 53% more time on malicious emails and did not simply confirm its malicious verdicts, but they were more likely to accept its benign verdicts, including planted false negatives.

Sep 2025
Aug 2025
Aug 8, 2025
AIxCC final: Team Atlanta wins as systems patch 43 of 54 found synthetic bugs and find 18 real ones
DefenseCompetitionDARPA, ARPA-H, Team Atlanta

DARPA reports that seven finalist cyber reasoning systems analyzed over 54 million lines of code, found 54 unique synthetic vulnerabilities in 63 challenges and patched 43, and found 18 real non-synthetic vulnerabilities with 11 patches. Team Atlanta won $4 million, Trail of Bits $3 million and Theori $1.5 million; DARPA and ARPA-H added $1.4 million for real-world integration and four systems were open-sourced on the day.

Aug 5, 2025
Microsoft's Project Ire agent autonomously reverse engineers and classifies malware
DefenseTool releaseMicrosoft

Microsoft Research describes Project Ire, a prototype LLM agent that uses decompilers and binary analysis tools to reverse engineer software and classify it as malicious or benign, producing an auditable chain-of-evidence report. Microsoft reports 0.98 precision and 0.83 recall on a Windows driver dataset, but 0.26 recall on about 4,000 hard real-world files, and plans to deploy it in Defender as Binary Analyzer.

Jun 2025
Jun 27, 2025
Paper proposes test and evaluation process with effectiveness metrics for RL cyber defence agents
DefensePaper

A paper in Applied AI Letters by QinetiQ researchers sets out a test and evaluation process for cyber defence agents covering performance, effectiveness, resilience and generalisability, and demonstrates its low-fidelity stage on CAGE Challenge 2 RL agents in CybORG. It introduces Measures of Effectiveness tailored to cyber defence alongside RL reward and tests agents under environment perturbations not seen in training.

May 2025
May 7, 2025
UC Santa Cruz study integrates LLM agents into CAGE 4 and finds RL defenders still outperform them
DefensePaperUC Santa Cruz

Researchers led by UC Santa Cruz integrated LLM agents into the CybORG CAGE 4 multi-agent defence environment and proposed a communication protocol for mixed LLM and RL teams. In their runs an all-RL team scored far better reward than an all-LLM (GPT-4o-mini) team and acted about 104 times faster, though the authors highlight LLM explainability and note the environment was designed for RL agents.

Mar 2025
Mar 24, 2025
Microsoft announces Security Copilot agents for phishing triage, alert triage and remediation
DefenseTool releaseMicrosoft, OneTrust, Aviatrix

Microsoft announced Microsoft-built Security Copilot agents, including a Phishing Triage Agent in Defender, alert triage agents in Purview, a Conditional Access Optimization Agent, a Vulnerability Remediation Agent in Intune and a Threat Intelligence Briefing Agent, plus five partner agents. Preview was planned from April 2025; the announcement contains no evaluation of agent accuracy.

Findings

Research openings

Benchmarks and tools