Jun 13, 2025
SEC-bench automatically builds real vulnerability tasks and finds agents patch at most 34%
SEC-bench uses multi-agent scaffolding to construct reproducible vulnerability instances with test environments and validated patches from real projects, at about $0.87 per instance. The authors report that LLM agents reached at most 18.0% on proof-of-concept generation and 34.0% on vulnerability patching.