SafetyTransparencyAccountabilitycs.AI
Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety
Tomek Korbak, Mikita Balesni, Elizabeth Barnes, Yoshua Bengio, Joe Benton, Joseph Bloom, Mark Chen, Alan Cooney, Allan Dafoe, Anca Dragan, Scott Emmons, Owain Evans, David Farhi, Ryan Greenblatt, Dan Hendrycks, Marius Hobbhahn, Evan Hubinger, Geoffrey Irving, Erik Jenner, Daniel Kokotajlo, Victoria Krakovna, Shane Legg, David Lindner, David Luan, Aleksander Mądry, Julian Michael, Neel Nanda, Dave Orr, Jakub Pachocki, Ethan Perez, Mary Phuong, Fabien Roger, Joshua Saxe, Buck Shlegeris, Martín Soto, Eric Steinberger, Jasmine Wang, Wojciech Zaremba, Bowen Baker, Rohin Shah, Vlad Mikulik
July 15, 2025
arXiv.org
270 citations
12 influential
Key Takeaways
-
Chain of thought monitoring offers a new window into AI reasoning: When AI systems "think out loud" in human language before giving answers, safety researchers can potentially detect whether the system intends to misbehave by examining these intermediate thoughts—a capability that didn't exist with previous AI systems.
-
This safety method is imperfect but still valuable: The paper acknowledges that CoT monitoring cannot catch all instances of misbehavior, but the authors argue it's promising enough to warrant investment alongside other existing safety approaches rather than as a replacement for them.
-
CoT monitorability is fragile and could be lost: The authors warn that certain AI development decisions could undermine or eliminate the ability to monitor chains of thought effectively, suggesting that frontier model developers need to actively consider how their design choices impact this safety opportunity.
-
This represents a unique opportunity that shouldn't be taken for granted: Unlike other known AI oversight methods, CoT monitoring is specifically enabled by AI systems that reason in human language, making it a distinctive tool that could disappear if development practices change without intentional preservation efforts.
Abstract
AI systems that "think" in human language offer a unique opportunity for AI safety: we can monitor their chains of thought (CoT) for the intent to misbehave. Like all other known AI oversight methods, CoT monitoring is imperfect and allows some misbehavior to go unnoticed. Nevertheless, it shows promise and we recommend further research into CoT monitorability and investment in CoT monitoring alongside existing safety methods. Because CoT monitorability may be fragile, we recommend that frontier model developers consider the impact of development decisions on CoT monitorability.