ExploitBench

Benchmark that separates crashes from real exploitation progress on browser-engine targets.

Records citing ExploitBench

Sep 29, 2026
Anthropic assesses open-weight GLM-5.3: exploit results near Mythos Preview, safeguards bypassed in 64% to 100% of simulated trials
CapabilityEvaluation reportAnthropic, US Center for AI Standards and Innovation, Moonshot AI

Anthropic's Frontier Red Team reports that Zhipu AI's open-weight GLM-5.3 developed end-to-end exploits in 50 of 410 ExploitBench attempts, against 56 of 410 for Claude Mythos Preview, and full control-flow hijacks in 4% of trials on a 100-task subset of Anthropic's internal binary-exploitation benchmark, against 6% for Mythos Preview. In a simulated harmful-request test, Anthropic reports the model's refusals were bypassed in 64% to 100% of trials using a cover story, prefilled reasoning, or an abliterated copy of the weights, techniques it says did not work on safeguarded Claude models. All figures are Anthropic's own and were produced on setups the post describes only in part.

Sep 22, 2026
Claude Opus 5.5 system card reports cyber results at or above Mythos 5.1 and a three-stage cyber safeguard
CapabilitySystem cardAnthropic, Irregular, US Center for AI Standards and Innovation

Anthropic's Claude Opus 5.5 system card reports that, evaluated through the API with its cyber safeguards off, the model meets or exceeds Claude Mythos 5.1 and Claude Opus 5 on every cyber evaluation it includes (ExploitBench, CyScenarioBench, a rewritten Binary Exploitation Benchmark and ExploitGym). Anthropic places the model in the lower category of its Frontier Compliance Framework and says it sees no indication of novel offensive capability. It deployed the model with a three-stage cyber safeguard (an activation probe and two classifiers) that blocks vulnerability discovery in compiled binaries, and reports no evidence of a critical-severity jailbreak from internal and contracted external red teaming (the card gives no CAISI results).

Jul 23, 2026
UK AISI and US CAISI jointly assess Kimi K3 cyber capability as trailing US frontier models
CapabilityEvaluation reportUK AI Security Institute, US Center for AI Standards and Innovation, Moonshot AI

The UK AI Security Institute and US CAISI published a joint preliminary assessment of Moonshot AI's open-weight Kimi K3. They report it trails leading US closed models on exploit development and a 32-step cyber range, and that its safeguards did not stop it attempting exploit development.

May 13, 2026
ExploitBench grades AI exploit development as a 16-step capability ladder on V8 bugs
CapabilityBenchmarkCarnegie Mellon University, Bugcrowd

Carnegie Mellon researchers released ExploitBench, which scores exploitation progress on 41 V8 JavaScript-engine vulnerabilities across 16 flags from reaching the bug through arbitrary read/write, control-flow hijack and code execution. The paper reports that public models routinely reach and crash vulnerable code but rarely achieve arbitrary code execution, while one private frontier model succeeded on roughly half of cases.