SafetyTransparencyAccountabilitycs.CY
AI Safety Frameworks Should Include Procedures for Model Access Decisions
Edward Kembery, Tom Reed
November 15, 2024
arXiv.org
Key Takeaways
-
Access decisions significantly shape AI risks and benefits: The paper emphasizes that how AI models are distributed—including who gets access and what type of access they receive—fundamentally determines whether a model helps or harms society, yet current decisions about this are made without clear, transparent procedures.
-
Current practices are ad hoc and lack evidence: Existing AI safety frameworks and company policies recognize that access levels matter for safety, but they make access decisions in ways that are opaque, inconsistent, and not grounded in empirical testing or data about actual risks.
-
Companies should establish transparent "Responsible Access Policies": The paper proposes that AI developers create formal, publicly documented procedures (called RAPs) for deciding who can access their models and under what conditions, moving away from behind-the-scenes decision-making.
-
Three essential components for access decisions: Responsible Access Policies should include: (1) empirical testing of what different users can actually do with a model under different access levels, (2) systematic assessment of which user groups pose higher or lower risks, and (3) clear advance commitments about when access will be granted or removed based on specific conditions.
-
Transparency and predictability improve accountability: By making access decisions procedural and pre-committed rather than case-by-case, companies can be held accountable for their choices and stakeholders can better understand and trust how powerful AI systems are being distributed.