AccountabilityTransparencyTrustworthinesscs.CY
Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims
Miles Brundage, Shahar Avin, Jasmine Wang, Haydn Belfield, Gretchen Krueger, Gillian Hadfield, Heidy Khlaaf, Jingying Yang, Helen Toner, Ruth Fong, Tegan Maharaj, Pang Wei Koh, Sara Hooker, Jade Leung, Andrew Trask, Emma Bluemke, Jonathan Lebensold, Cullen O'Keefe, Mark Koren, Théo Ryffel, JB Rubinovitz, Tamay Besiroglu, Federica Carugati, Jack Clark, Peter Eckersley, Sarah de Haas, Maritza Johnson, Ben Laurie, Alex Ingerman, Igor Krawczuk, Amanda Askell, Rosario Cammarota, Andrew Lohn, David Krueger, Charlotte Stix, Peter Henderson, Logan Graham, Carina Prunkl, Bianca Martin, Elizabeth Seger, Noa Zilberman, Seán Ó hÉigeartaigh, Frens Kroeger, Girish Sastry, Rebecca Kagan, Adrian Weller, Brian Tse, Elizabeth Barnes, Allan Dafoe, Paul Scharre, Ariel Herbert-Voss, Martijn Rasser, Shagun Sodhani, Carrick Flynn, Thomas Krendl Gilbert, Lisa Dyer, Saif Khan, Yoshua Bengio, Markus Anderljung
April 15, 2020
arXiv.org
512 citations
33 influential
Key Takeaways
-
Verifiable claims are essential for trust: The paper argues that AI developers must make specific, accountable claims about their systems' safety, security, fairness, and privacy—and be able to back them up with evidence, rather than relying on general assurances or self-regulation.
-
Current safeguards are inadequate: Existing regulations and industry norms are insufficient to ensure responsible AI development, indicating that new mechanisms and accountability structures are needed across the AI development lifecycle.
-
Multiple stakeholders need scrutiny tools: Users, customers, civil society, governments, and other groups require effective means to independently verify AI developers' claims, suggesting that transparency mechanisms should be designed for diverse audiences, not just technical experts.
-
Ten mechanisms span institutions, software, and hardware: The paper identifies a range of approaches—from organizational practices to technical tools to physical safeguards—indicating that trustworthy AI requires coordinated action across multiple levels rather than single solutions.
-
Focus on four key dimensions: The paper emphasizes that verifiable claims should specifically address safety, security, fairness, and privacy protection, suggesting these are the primary areas where stakeholders need concrete evidence of responsible development practices.