Methods/Defense

Independent patch verification

Checking AI-generated patches with tests beyond the one that motivated them, before accepting them.

4 records4 defense1 findings (1 measured)First recorded 2025-04assistant-drafted

How it works

Additional security variants, regression tests, and reviews catch patches that stop a crash without fixing the underlying flaw.

What we know

1 corroborated

Records over time

RangeLanes
4 of 4 records in view

Use the arrow keys to move between records, Home and End to jump to the first and last, and Enter to select one.

Agents find real bugsAgents in real operationsGated capability, incidents in the labAttackCapabilityDefensePolicyJan 25Jul 25Jan 26Jul 26
Full record · drag to choose a range
2026

Select a mark to read the record. Mark size shows editorial significance. Hollow marks are dated to the month. Era bands are editorial labels.

Records in view

4 records · newest first
Mar 2026
Mar 9, 2026
OSS-CRS makes AIxCC reasoning systems runnable locally; OpenSSF adopts it as a sandbox project
DefenseTool releaseGeorgia Institute of Technology, Microsoft, Team Atlanta

Researchers led by Georgia Tech released OSS-CRS, a locally deployable framework for running and combining AIxCC cyber reasoning systems, noting that all seven open-sourced finalist systems depended on competition cloud infrastructure that no longer exists. Porting the winning Atlantis system, they found 10 previously unknown bugs (three high severity) in 8 OSS-Fuzz projects; OpenSSF welcomed OSS-CRS into its AI/ML Security Working Group in April 2026.

Feb 2026
Feb 20, 2026
Anthropic releases Claude Code Security in limited preview to scan code and propose patches
DefenseTool releaseAnthropic

Anthropic released Claude Code Security as a limited research preview for Enterprise and Team customers, with expedited free access for open-source maintainers. The tool reasons about data flow across a codebase, re-examines each finding in a multi-stage verification pass, assigns severity and confidence ratings, and proposes patches that are applied only with human approval.

Feb 7, 2026
AIxCC SoK finds stability decided results and many validated AI patches were still semantically wrong
DefensePaperGeorgia Institute of Technology, Texas A&M University, DARPA

A systematization-of-knowledge paper by organizers and competitors analyzes AIxCC's design, the seven finalist architectures and results beyond the scoreboard. It reports that system stability and accuracy penalties decided rankings, that LLM-based systems found vulnerabilities a fuzzing baseline missed, and that among patches passing all automatic validation, manual review found semantic errors in 38-46% from baseline agents; the top two systems had 83.8% and 79.2% competition-scored patch accuracy.

Apr 2025
Apr 29, 2025
Meta releases AutoPatchBench to test AI repair of fuzzing-found C/C++ vulnerabilities
DefenseBenchmarkMeta

Meta introduced AutoPatchBench, part of CyberSecEval 4, with 136 fuzzing-identified C/C++ vulnerabilities and verified fixes, plus a 113-case Lite subset with single-function root causes. Patches are checked by build and crash reproduction, then fuzzing and white-box differential testing; Meta's reference agent generated crash-stopping patches in about 60% of cases, but only 5-11% passed the stricter checks.

All records