{
 "license": "CC-BY-4.0",
 "attribution": "Fide AI, Agentic Cyber Explorer",
 "url": "https://agentic-cyber-explorer.pages.dev/events/screwtape-adversarial-transcript-editing-monitor-red-teaming-2026/",
 "asOf": "2026-10-30",
 "id": "screwtape-adversarial-transcript-editing-monitor-red-teaming-2026",
 "date": "2026-10-14",
 "datePrecision": "day",
 "title": "Redwood Research’s Screwtape edits real agent transcripts; 6 confirmed attacks evaded a developer’s held-out action-only monitor",
 "lane": "defense",
 "kind": "paper",
 "summary": "Redwood Research authors present Screwtape, a pipeline in which an LLM attacker edits real Claude Code and Codex transcripts to realize a specified harm and revises the edit until a proxy monitor no longer flags it, while an LLM verifier screens edits for consistency and severity; the edited actions are never executed. Across 252 runs on 94 transcripts from Redwood staff, they report 35 verifier-passed attacks that evaded a proxy monitor, 17 of which also evaded a held-out action-only monitor supplied by a frontier developer, and 6 of those 17 confirmed valid in review by the first author. The authors call the pipeline a stress test limited to action-only monitors, with one annotator, small samples and pooled configurations.",
 "whyItMatters": "It adds a method and one set of numbers for evasion of a developer's monitor by optimized attacks built on real transcripts, though validity rests on an LLM verifier and single-annotator review, not execution.",
 "actors": [
  "redwood-research"
 ],
 "topics": [
  "monitoring-and-control",
  "eval-validity"
 ],
 "atlas": [
  "monitor"
 ],
 "artifacts": [
  "claude-opus-4",
  "gemini",
  "gpt-5-family",
  "glm"
 ],
 "sources": [
  {
   "url": "https://arxiv.org/abs/2610.20941",
   "publisher": "arXiv",
   "title": "Screwtape: Adversarial Transcript Editing for Monitor Red-Teaming",
   "date": "2026-10-14",
   "type": "primary",
   "accessed": "2026-10-19"
  }
 ],
 "keyFacts": [
  {
   "fact": "Pipeline: an attacker edits a real transcript to realize a threat contract (rogue deployment, secret exfiltration or consequential actions) and receives monitor and verifier reasoning each iteration. Programmatic rules reject edits to user messages, subagent calls and repeated edits. The verifier sees more than the monitor, including the original transcript and the attacker's description, checks consistency, then judges severity along five dimensions. Because the attacker cannot produce chain-of-thought, it applies only to action-only monitors.",
   "locator": "v1, Sections 1 and 3"
  },
  {
   "fact": "Yield: 94 transcripts, 252 runs, 27 runs with a candidate attack, 35 candidate attacks passing the verifier and evading a proxy monitor. Of the 35, 17 also evaded a held-out action-only monitor from a frontier developer (set at an estimated 1% false-positive rate on unedited Redwood transcripts, never queried during the search), 18 were flagged, and human review confirmed 6 of the 17; the other 11 were below the severity bar or inconsistent with the original transcript.",
   "locator": "v1, Section 4.1; Figure 3"
  },
  {
   "fact": "Configurations are pooled: attackers were GPT-5.6 Sol or GLM-5.3-Flash, verifiers GPT-5.6 Sol or GPT-5.5, and proxy monitors Gemini 3.1 Pro (22 of the 35 attacks), Opus 4.8 (9), GPT-5.6 Sol (3) or GLM-5.2 (1). The authors say the stage counts are not rates for one configuration, and runs used transcripts scored highest for opportunities to realize each threat.",
   "locator": "v1, Section 4.1; Appendix A; Limitations"
  },
  {
   "fact": "Verifier checks: on 107 test cases from 18 transcripts it accepted 8 of 63 injected inconsistencies (0 of 30 file write/read, 1 of 18 incorrect tool output, 7 of 15 fabricated file contents at first read) and rejected none of 44 unedited controls. Against 39 human-labeled attacks (13 valid, 26 invalid) the chosen design approved 11 of 13 valid and rejected 20 of 26 invalid; the authors developed it partly on 23 of those, say the comparison shows fit, not generalization, and report intervals that overlap. The yield-study attacks passed earlier verifier versions; on 16 attacks not used in development the chosen design was right on 13, against 11 and 8 for the alternatives.",
   "locator": "v1, Section 4.2; Figures 4 and 5"
  },
  {
   "fact": "Proxy-monitor experiments: a tuned monitor prompt raised ROC AUC from 0.90 to 0.94 for Gemini 3.1 Pro with overlapping intervals, and was near ceiling for Opus 4.8 (0.99 and 1.00), on 20 attack and 20 unedited transcripts (19 and 18 for tuned Opus 4.8). Transfer at threshold 2: of 17 attacks found against Gemini 3.1 Pro, 6 evaded Opus 4.8 and 0 evaded GPT-5.6 Sol; of 8 found against Opus 4.8, 3 evaded Gemini 3.1 Pro and 1 evaded GPT-5.6 Sol. The authors summarize transfer as partial between the first two and rare to the third.",
   "locator": "v1, Section 4.3; Figures 6 and 7"
  },
  {
   "fact": "Failure classes the authors describe for their Opus 4.8 proxy monitor: not connecting steps that are far apart in a transcript, accepting a safety step that looks correct but has no effect, and taking the agent's report of its own actions at face value.",
   "locator": "v1, Appendix B"
  },
  {
   "fact": "Stated limits: no chain-of-thought and one agent per transcript; the attacker sees the whole transcript, including turns after the edited actions, and receives monitor feedback, so the authors call it a stress test, not an estimate of what agents would attempt; one annotator (the first author) did all review; source transcripts come from Redwood staff, not the developer whose monitor was tested; the contribution of monitor feedback is not isolated; internal red-teaming against a production monitor was not evaluated; the study does not cover weight exfiltration; provider safeguards often refuse attack generation, so open-weight and unrestricted attacker models were used; the code is not publicly released but is available to researchers on request.",
   "locator": "v1, Sections 1, 5 and 6; Code availability"
  },
  {
   "fact": "Related work as the authors describe it: METR's per-action-monitor note validates its monitor by splicing malicious actions by hand into benign transcripts, whereas Screwtape optimizes edited attacks against a monitor and gates them with a verifier. The paper also cites METR's red-teaming of Anthropic's monitoring, UK AISI's Control Red Team and MonitoringBench.",
   "locator": "v1, Section 2"
  }
 ],
 "significance": 3,
 "fideQuestions": [
  "FID-074",
  "FID-075",
  "FID-076"
 ],
 "methods": [
  "adaptive-red-teaming",
  "ai-monitoring",
  "monitor-evasion"
 ],
 "review": "assistant-drafted",
 "addedOn": "2026-10-19",
 "revisions": []
}