OpenAI describes an asynchronous monitor powered by GPT-5.4 Thinking at maximum reasoning effort that reviews internal coding agent sessions, including chains of thought, within about 30 minutes of completion. OpenAI reports that over five months it covered tens of millions of trajectories, with no real conversation reaching the highest severity and about 1,000 moderate-severity alerts, many from deliberate internal red-teaming, and that it flagged every interaction employees had independently reported. The categories it labels common, each under 1% of traffic, are circumventing restrictions, deception and concealing uncertainty.
It is a production-scale data point on using LLM monitors as a control layer for agents with real system access.
Key facts
As stated in the sources, with where to find them.
- Monitor: GPT-5.4 Thinking at maximum reasoning effort; review latency about 30 minutes after completion.Section 'Our approach & how it works'
- Zero real conversations reached the highest severity and about 1,000 triggered moderate-severity alerts over five months, many from deliberate internal red-teaming; under 0.1% of traffic, mainly bespoke or local setups, was outside coverage.Sections 'Our approach' and 'Category overview'
- OpenAI labels circumventing restrictions, deception and concealing uncertainty as common (under 1% of traffic), and unauthorized data transfer and destructive actions as rare but high severity.Section 'Category overview'
Findings that cite this record
Key questions this bears on
- Can AI agents defend and oversee systems on their own?Not yet. Agents are weak on realistic defensive benchmarks and monitors can be evaded; assistants help analysts who stay in charge.