AccountabilityEthicsSafetycs.SE
Towards Concrete and Connected AI Risk Assessment (C$^2$AIRA): A Systematic Mapping Study
Boming Xia, Qinghua Lu, Harsha Perera, Liming Zhu, Zhenchang Xing, Yue Liu, Jon Whittle
January 27, 2023
2023 IEEE/ACM 2nd International Conference on AI Engineering – Software Engineering for AI (CAIN)
25 citations
2 influential
Key Takeaways
-
Fragmentation across frameworks: The study analyzed 16 AI risk assessment frameworks from industry, governments, and NGOs, revealing significant divergence in how different organizations approach AI risk assessment, making it challenging for stakeholders to select appropriate frameworks for their specific contexts.
-
Inconsistent coverage of responsible AI principles: The frameworks show varying levels of attention to key responsible AI principles, suggesting that some critical areas of concern may be inadequately addressed depending on which framework an organization adopts.
-
Lifecycle and stakeholder gaps: The analysis identified that existing frameworks differ substantially in which stages of AI system development they address and which stakeholders they engage, indicating that some important phases or perspectives may be overlooked in certain frameworks.
-
Need for convergence and standardization: The study highlights both areas of convergence and divergence among frameworks, suggesting that greater alignment and standardization could help organizations more effectively and consistently assess AI risks across different contexts and domains.
-
Domain and geographical limitations: The frameworks show variation in their applicability across different sectors and geographical regions, pointing to the need for either more universally applicable frameworks or better guidance on adapting existing ones to specific local and industry contexts.