AccountabilitySafetyTrustworthinesscs.CY
Evolving AI Risk Management: A Maturity Model based on the NIST AI Risk Management Framework
Ravit Dotan, Borhane Blili-Hamelin, Ravi Madhavan, Jeanna Matthews, Joshua Scarpino
January 26, 2024
arXiv.org
30 citations
9 influential
Key Takeaways
-
There's a significant gap between AI risk management principles and real-world practice: While frameworks like NIST's AI Risk Management Framework represent emerging consensus on best practices, the paper finds that private sector organizations "lag far behind" in implementation, with efforts that are "sporadic and selective at best."
-
"Responsible AI" can be performative without substance: The paper warns that incomplete implementation of risk management frameworks can create a "misleading veneer of trustworthy processes," where organizations appear to follow responsible AI practices while actually engaging in "substantively harmful practices"—essentially using compliance as cover rather than genuine protection.
-
The field is shifting from abstract principles to measurable practices: The research reflects a broader movement in responsible AI away from vague ethical principles toward "tangible and operationalizable practices" that can actually mitigate real harms, suggesting that meaningful progress requires concrete, implementable standards rather than aspirational statements.
-
A maturity model can help organizations assess their readiness: The paper proposes using a flexible maturity model based on the NIST AI RMF to evaluate "where organizations sit relative to the emerging consensus," providing a tool for stakeholders to measure progress and identify gaps in sociotechnical harm mitigation efforts.