Not yet. Agents are weak on realistic defensive benchmarks and monitors can be evaded; assistants help analysts who stay in charge.
LLM agents perform poorly on realistic threat-hunting and investigation benchmarks, while vendor-run trials report assistants speeding up analysts who remain in charge. Agent monitors can be driven to miss covert actions under optimized attack, so catch rates measured against ordinary mistakes say little about adversarial settings.
The findings behind it
2 corroborated, 1 reported, 1 qualifiedEach finding carries a status that changes as new work arrives. What the statuses mean.
- LLM agents fall well short of reliable performance on realistic threat-investigation and threat-hunting benchmarks built from security logs.Corroborated measured · 3 evidence records
- Controlled trials run by Microsoft report that its security assistants make analysts faster and more accurate.Reported reported · 2 evidence records
- Agent monitors can be driven to miss covert actions: optimized attacks pushed suspicion scores near zero, and weak red-teaming overstates catch rates.Corroborated measured · 3 evidence records
- OpenAI reports its internal coding-agent monitor matched every staff escalation, and OpenAI and Google DeepMind report that most flags reflect overeagerness or mistakes rather than adversarial intent.Qualified reported · 2 evidence records
Answer history
Answers are never edited after the fact. A revision adds a new answer and keeps the earlier ones here.
- 2026-09-25moderate confidencecurrentNot yet. Agents are weak on realistic defensive benchmarks and monitors can be evaded; assistants help analysts who stay in charge.First answer, drawn from the findings linked here.