SafetyAccountabilityTransparencycs.CY
International Agreements on AI Safety: Review and Recommendations for a Conditional AI Safety Treaty
Rebecca Scholefield, Samuel Martin, Otto Barten
March 18, 2025
arXiv.org
7 citations
Key Takeaways
-
International consensus is emerging on AI safety concerns: The paper reviews multiple 2023+ proposals for international AI agreements and identifies significant areas of agreement among experts, suggesting that despite differing approaches, there is growing recognition that advanced AI systems pose risks serious enough to warrant international coordination.
-
A "compute threshold" approach could be practically implementable: Rather than trying to regulate all AI development, the proposed treaty would focus oversight specifically on systems developed using computing power above a certain threshold, making it more feasible to monitor and enforce than broader regulations.
-
Multiple verification mechanisms are needed together: The paper recommends combining audits of AI models themselves, information security practices, and governance structures—overseen by an international network of AI Safety Institutes—rather than relying on any single verification method, recognizing that comprehensive oversight requires checking multiple dimensions.
-
International AI Safety Institutes would need real enforcement power: A key feature of the proposal is that these institutes would have authority to pause development if risks are deemed unacceptable, meaning the agreement would need to establish genuine enforcement mechanisms rather than serving as a non-binding framework.
-
The proposal balances immediate action with long-term flexibility: The treaty design includes measures that can be implemented now while maintaining a structure that can adapt as AI research evolves and our understanding of risks improves, addressing the challenge that both the technology and our knowledge about it are rapidly changing.
Abstract
The malicious use or malfunction of advanced general-purpose AI (GPAI) poses risks that, according to leading experts, could lead to the 'marginalisation or extinction of humanity.' To address these risks, there are an increasing number of proposals for international agreements on AI safety. In this paper, we review recent (2023-) proposals, identifying areas of consensus and disagreement, and drawing on related literature to assess their feasibility. We focus our discussion on risk thresholds, regulations, types of international agreement and five related processes: building scientific consensus, standardisation, auditing, verification and incentivisation. Based on this review, we propose a treaty establishing a compute threshold above which development requires rigorous oversight. This treaty would mandate complementary audits of models, information security and governance practices, overseen by an international network of AI Safety Institutes (AISIs) with authority to pause development if risks are unacceptable. Our approach combines immediately implementable measures with a flexible structure that can adapt to ongoing research.
34 pages
Paper Info
arXiv ID2503.18956
PublishedMarch 18, 2025
Citations
7
Relevance95%
VenuearXiv.org
TypeJournalArticle, Review
Categorycs.CY
Responsible AI Topics
SafetyAccountabilityTransparency
Links
Citation data from Semantic Scholar