Fundamental Impossibility Results in AI Safety: Trading Off between Desiderata

Safety goals may inherently conflict: The paper demonstrates a fundamental tradeoff between key AI safety objectives—specifically between making AI systems helpful and keeping them harmless. This suggests that achieving all desirable safety properties simultaneously may be mathematically impossible rather than simply difficult.

Public information creates unavoidable vulnerabilities: When AI systems provide advice publicly (accessible to everyone), that same helpful information can be exploited by bad actors. The paper formalizes this as a game-theoretic problem, showing that defenders cannot benefit from public AI guidance without also enabling potential attackers to exploit it.

Information control has limits: The research explores whether AI systems could selectively share information to help defenders while protecting against attackers, but concludes this approach has fundamental constraints. This suggests that simply "hiding" information from bad actors while helping good actors is not a reliable safety strategy.

Computational limitations matter for safety: The paper models defenders and attackers as having bounded computational resources, suggesting that real-world AI safety depends not just on what information exists, but on what different actors can actually process and exploit given their technical capabilities.

Tradeoffs should inform AI design: Rather than searching for a perfect solution, the authors suggest these fundamental impossibility results should guide how we design safer AI systems—by explicitly acknowledging and managing the inherent tensions between safety goals rather than pretending they don't exist.