Artificial Analysis announced a Cyber Index Alliance, with Collinear AI, IBM, NVIDIA and Vercel as launch partners, and a v1 Cyber Index that combines three source-code evaluations: repository audit and patch (CWE-Bench-AA, 120 private tasks), finding expert-verified vulnerabilities (DeepsecBench-AA) and discovering, reproducing and patching memory-safety bugs (CyberGym-E2E-AA, 131 tasks). Artificial Analysis says it excludes exploit building, runs all three on its open-source Stirrup harness, and reports safety refusals separately from scores. The article reports failure-mode statistics and says several frontier models refuse almost every CyberGym-E2E-AA task; per-model index scores appear only in charts.
IBM
Sep 28, 2026
Artificial Analysis launches a Cyber Index and industry alliance scoring models on source-level vulnerability finding and fixing
Jun 10, 2025
Researchers from labs and industry publish design patterns for provably injection-resistant agents
Authors from Invariant Labs, IBM, ETH Zurich, Swisscom, Google, Microsoft and others propose six design patterns that limit what injected text can do, such as action-selector, plan-then-execute, dual LLM, LLM map-reduce, code-then-execute and context minimization. They discuss utility and security trade-offs and illustrate the patterns with ten application case studies.