Reasons to Doubt the Impact of AI Risk Evaluations

Evaluations may not translate understanding into action: The paper identifies four specific ways that improved understanding of AI risks doesn't necessarily lead to better risk mitigation, including "challenges in upholding and enforcing commitments"—meaning that even when labs understand risks, they may not follow through on addressing them.

Real-world performance matters more than lab tests: One key failure mode is that "risks manifest beyond the AI system or insignificant returns from evaluations compared to real-world observations," suggesting that controlled evaluations in labs may miss critical risks that only emerge when AI systems are deployed in the real world.

Evaluations could backfire: The paper warns that evaluations could be actively harmful by "triggering the weaponization of dual-use capabilities," meaning that detailed safety testing could inadvertently help bad actors weaponize AI technology, or by creating "high opportunity costs for AI safety" by diverting resources from more effective approaches.

Current evaluation practices need strategic rethinking: Rather than assuming more evaluations are always better, the paper argues for "a more strategic and impactful approach to AI risk assessment and mitigation," with 12 specific recommendations for labs, regulators, and researchers to make evaluations more effective.

The value proposition of AI safety evaluations is unproven: The core finding questions whether the substantial resources invested in AI system evaluations actually "significantly improve our understanding of AI risks," suggesting the field may be overestimating how much current evaluation practices contribute to safety.