SafetyAccountabilityTransparencycs.AI
The AI risk repository: A meta-review, database, and taxonomy of risks from artificial intelligence
Peter Slattery, Alexander K. Saeri, Emily A. C. Grundy, Jess Graham, Michael Noetel, Risto Uuk, James Dao, Soroush Pour, Stephen Casper, Neil Thompson
August 14, 2024
Patterns
25 citations
1 influential
Key Takeaways
-
Terminology chaos is a real problem: The paper analyzed 74 AI risk frameworks containing 1,725 distinct risks and found that researchers use different terms for the same problems (like "specification gaming" and "reward hacking" describing identical issues), making it difficult for organizations to compare findings and ensure comprehensive risk coverage.
-
Human decisions cause as many AI risks as the AI itself: Contrary to common assumptions that focus blame on AI systems, the research found that human decisions are responsible for 38% of AI risks while the AI systems themselves cause 42%—highlighting that responsible AI requires attention to how humans design, deploy, and oversee these systems.
-
We need a shared language for AI safety: The lack of standardized terminology creates "friction" that slows down progress on AI safety, making it harder for developers, policymakers, and auditors to communicate about risks and learn from each other's work across different organizations and countries.
-
This is a practical tool, not just academic: The paper provides a unified classification system designed for real-world use by developers conducting risk assessments, policymakers writing regulations, and auditors evaluating AI systems—making AI risk discussion more concrete and actionable.
Abstract
Artificial intelligence (AI) is reshaping society, from video generation to medical diagnosis, coding agents to autonomous vehicles. Yet researchers, policymakers, and technology companies lack shared terminology for discussing AI risks. Consider "privacy": one framework uses this term to describe a model's ability to leak sensitive training data, while another uses it to mean freedom from government surveillance. Conversely, researchers have introduced "Goodhart's law," "specification gaming," "reward hacking," and "mesa-optimization" to describe the same phenomenon of AI systems optimizing for measured proxies rather than intended goals. This terminological diversity creates friction: comparing findings across studies requires mapping between frameworks, and comprehensive risk coverage requires consulting multiple taxonomies that use different organizing principles. This paper addresses this challenge by creating a comprehensive catalog of AI risks. We systematically analyzed every major AI risk framework published to date-74 frameworks containing 1,725 distinct risks-and organized them into a unified system. Our two classification systems reveal important patterns: contrary to common assumptions, human decisions cause nearly as many AI risks (38%) as the AI systems themselves (42%). The work provides practical tools for anyone working on AI safety, from developers conducting risk assessments to policymakers writing regulations to auditors evaluating AI systems. By establishing a common reference point, this repository creates the foundation for more coordinated and comprehensive approaches to managing AI's risks while realizing its benefits.