SafetySecurityTransparencycs.CY
Effective Mitigations for Systemic Risks from General-Purpose AI
Risto Uuk, Annemieke Brouwer, Tim Schreier, Noemi Dreksler, Valeria Pulignano, Rishi Bommasani
November 14, 2024
arXiv.org
8 citations
Key Takeaways
-
Three mitigation measures emerged as most effective: Safety incident reports and security information sharing, third-party pre-deployment model audits, and pre-deployment risk assessments all received expert agreement ratings above 60% across different risk categories and were selected in over 40% of experts' preferred mitigation combinations.
-
External scrutiny and transparency are foundational: The 76 surveyed experts—spanning AI safety, critical infrastructure, democratic processes, CBRN risks, and bias—identified external scrutiny, proactive evaluation, and transparency as key principles for successfully mitigating systemic risks from general-purpose AI.
-
A diverse toolkit of mitigations exists: Among 27 mitigation measures identified through literature review, experts perceived a broad range as both effective and technically feasible, suggesting that no single solution addresses systemic risks and a combination of approaches is necessary.
-
Expert consensus spans multiple risk domains: The high agreement ratings (>60%) for top mitigations across all four risk areas indicate that certain safeguards—particularly third-party audits and pre-deployment assessments—are viewed as valuable regardless of whether the concern is infrastructure, democracy, CBRN, or discrimination risks.
-
Implementation gaps remain: While the research identifies promising mitigations and provides policy recommendations, it highlights that previous frameworks have left gaps in understanding how effective these measures actually are in practice, suggesting more work is needed to translate expert consensus into real-world deployment.