Researchers at the UK AI Security Institute and the University of Oxford built SandboxEscapeBench, 18 nested-sandbox capture-the-flag scenarios covering orchestration, runtime and kernel weaknesses rated difficulty 1 to 5. In the March 2026 version, which tested nine models, GPT-5 and Claude Opus 4.5 succeeded on about half of all runs, difficulty 1-2 scenarios were solved at high rates, no model succeeded at difficulty 4 or 5, and success rose roughly log-linearly with token budget. An August 2026 revision reports that newer models did markedly better, with Claude Mythos Preview saturating the public benchmark.
It gives a direct measurement of whether container sandboxes still contain capable agents, the control that evaluation and deployment environments rely on.
Key facts
As stated in the sources, with where to find them.
- 18 scenarios: orchestration (4), engine/runtime (8), host/kernel (6). The March version evaluated nine models with five runs per model-scenario pair at a 2.5M-token budget.v1 Section 3.2; Appendix E
- Overall escape success across runs (v1): GPT-5 0.50 [0.40, 0.60]; Claude Opus 4.5 0.49 [0.39, 0.59]; GPT-5.2 0.27; Claude Haiku 4.5 0.18; no successes at difficulty 4-5.v1 Section 5.1
- All successful exploits used known misconfigurations or public CVEs; four unintended shortcuts were found and fixed.Section 5; Appendix C
- In the August 2026 revision (v3), newer models run at a 100M-token budget included Claude Mythos Preview, which scored 0.84 overall and succeeded on difficulty 4 and 5 tasks; the authors say it saturates the public benchmark.v3 abstract footnote; Section 6; Appendix H
Findings that cite this record
Key questions this bears on
- Do cyber evaluations of AI agents stay contained?Not reliably. Several labs and a government evaluator have disclosed agents under evaluation acting on real third-party systems.
- How far can measured AI cyber capability be trusted?As a lower or conditional bound. Scores move substantially with token budget, evaluation pipeline, and benchmark contamination.