Agentic Cyber Explorerby Fide AI · PreviewPreview
Benchmarks and tools/benchmark

SEC-bench

Automatically constructed benchmark of real vulnerability reproduction and patching tasks.

Records citing SEC-bench

Jun 13, 2025
SEC-bench automatically builds real vulnerability tasks and finds agents patch at most 34%
DefenseBenchmarkUniversity of Illinois Urbana-Champaign, Purdue University

SEC-bench uses multi-agent scaffolding to construct reproducible vulnerability instances with test environments and validated patches from real projects, at about $0.87 per instance. The authors report that LLM agents reached at most 18.0% on proof-of-concept generation and 34.0% on vulnerability patching.