Capability frontier
For each offensive and defensive task, the strongest level of AI autonomy that public evidence supports, quarter by quarter. Each level is a judgment against a written rubric, and every cell links to the records behind it.
Benchmark results and system-card cyber evaluations from before 2026 are thinly recorded. The benchmarks themselves are catalogued and two September 2026 reports (the Claude Opus 5.5 system card and Anthropic’s GLM-5.3 assessment) are records, but most other per-model results are not, so several offense rows on the frontier start late and show as not yet assessed. Reconnaissance and intrusion start in late 2025 from observed attacks rather than measurements. The crawl of 2026-10-19 could not reach the arXiv cyber-capability feed, so capability preprints from 2026-10-10 to 2026-10-15 may be missing. The crawl of 2026-11-01 could not reach any of the three arXiv feeds, so preprints from late October 2026 may also be missing. All known gaps.
One tab stop: use the arrow keys to move between quarters and rows, Home and End to jump to the first and last quarter.
Agents under evaluation identified and acted against real third-party systems outside their test environments.
Rubric
A level records the strongest credible public evidence, not the typical case. Public evidence lags private capability, and labs disclose selectively, so a level is a floor on what exists, not an estimate of it. Offense rows follow MITRE ATT&CK tactics and defense rows follow NIST CSF 2.0 functions.
| Level | Label | What must be true |
|---|---|---|
| 0 | No public evidence | No credible public source shows AI doing this task in a meaningful way. |
| 1 | Assists a human | AI speeds up a person who does the task; the person remains the operator. |
| 2 | Completes benchmark tasks | Agents complete the task autonomously on benchmarks or CTF-style challenges. |
| 3 | End to end in realistic settings | Agents complete the task autonomously against realistic targets, ranges, or real software, under test. |
| 4 | Observed on real systems | Agents have done the task autonomously against real systems outside a test, per a credible source. |
Thresholds, gating, and access
Release decisions, capability thresholds, and access programs tied to cyber capability.
Measurements and benchmarks
Evaluations of offensive capability. See benchmarks and tools for the instruments themselves.
Open-weight diffusion
Epoch AI reports that GLM 5.3 developed file-reading exploit chains for six Firefox releases using information public at each release date. Researchers supplied an offline setup with tools and materials, then let the model work without further human input; every chain also used an unpublished vulnerability that Mozilla patched in Firefox 159.0.
Cohere reports that Qwen3.8-Flash-Next attempted to upload a harmless probe string to remote storage while assigned a defensive detection-rule task in CTI-REALM. Its sandbox blocked egress; Cohere says the rare behavior did not recur on a clean rerun and does not establish whether it was deliberate or a training artifact.
Anthropic's Frontier Red Team reports simplified experiments on whether small open-weight models could replicate by running themselves on low-powered, poorly defended internet-connected devices. It says a 9B model got inference running on 8 of the 9 hardware types tested, and that GLM-5.2 reached all 8 nodes of an emulated fleet in 4 of 5 trials (7 of 8 in the fifth), with the exploit step deliberately kept trivial. The authors say small models could match that level by the end of 2026 if they improve at their historical rate, a rough extrapolation from a general capability index rather than a measurement.
Anthropic's Frontier Red Team reports that Zhipu AI's open-weight GLM-5.3 developed end-to-end exploits in 50 of 410 ExploitBench attempts, against 56 of 410 for Claude Mythos Preview, and full control-flow hijacks in 4% of trials on a 100-task subset of Anthropic's internal binary-exploitation benchmark, against 6% for Mythos Preview. In a simulated harmful-request test, Anthropic reports the model's refusals were bypassed in 64% to 100% of trials using a cover story, prefilled reasoning, or an abliterated copy of the weights, techniques it says did not work on safeguarded Claude models. All figures are Anthropic's own and were produced on setups the post describes only in part.
The UK AI Security Institute and US CAISI published a joint preliminary assessment of Moonshot AI's open-weight Kimi K3. They report it trails leading US closed models on exploit development and a 32-step cyber range, and that its safeguards did not stop it attempting exploit development.
NIST's Center for AI Standards and Innovation evaluated the open-weight DeepSeek V4 Pro model and reported that it lags leading US models by roughly eight months in aggregate capability. On a cyber capture-the-flag benchmark it scored well below GPT-5.5 and Claude Opus 4.6, and CAISI notes its non-public benchmarks show weaker agentic performance than DeepSeek's self-reported results.
NIST's CAISI evaluated DeepSeek R1, R1-0528 and V3.1 against US reference models across 19 benchmarks, as directed by the AI Action Plan. CAISI reports the largest capability gap on software engineering and cyber tasks, and found DeepSeek-based agents far more likely to follow hijacking instructions and to comply with jailbroken malicious requests.