Agentic Cyber Explorerby Fide AI · PreviewPreview
Benchmarks and tools/benchmark

Agent Security Bench

Benchmark of attacks and defenses across LLM agent backbones and scenarios.

Records citing Agent Security Bench

Oct 3, 2024
Agent Security Bench formalizes attacks and defenses across ten LLM agent scenarios
DefenseBenchmarkarXiv

Zhang and colleagues release Agent Security Bench (ASB), covering 10 scenarios, 10 agents and over 400 tools, and benchmark attack and defense methods, including prompt injection, memory poisoning and a new backdoor attack, across 13 LLMs (23 methods in the first version, 27 in the ICLR 2025 version). They report a highest average attack success rate of 84.30% and limited effectiveness of current defenses. The paper was accepted at ICLR 2025.