AccountabilityTransparencySafetycs.AI
Coordinated Flaw Disclosure for AI: Beyond Security Vulnerabilities
Sven Cattell, Avijit Ghosh, Lucie-Aimée Kaffee
February 10, 2024
AAAI/ACM Conference on AI, Ethics, and Society
16 citations
1 influential
Key Takeaways
-
AI harm reporting currently lacks structure: Unlike cybersecurity, which has established Coordinated Vulnerability Disclosure (CVD) processes, AI systems rely on ad-hoc approaches to reporting algorithmic flaws, creating inconsistency in how problems are identified and addressed.
-
Machine learning requires specialized disclosure approaches: The paper argues that ML models present unique challenges that differ from traditional software vulnerabilities, necessitating a tailored framework rather than simply adopting existing cybersecurity practices.
-
The proposed Coordinated Flaw Disclosure (CFD) framework includes specific innovations: Key features include extended model cards for documentation, dynamic scope expansion to address emerging issues, independent adjudication panels for impartial review, and automated verification processes to validate reported flaws.
-
Structured disclosure could build public trust: By establishing transparent, organized processes for identifying and addressing AI flaws—rather than relying on informal reporting—CFD aims to improve accountability and demonstrate that organizations take algorithmic problems seriously.
-
Real-world testing is underway: The authors indicate a pilot program is forthcoming, suggesting this framework is moving from theory toward practical implementation in actual AI systems.