Towards a Responsible AI Development Lifecycle: Lessons From Information Security

Fairness metrics alone are insufficient: The paper finds that fairness metrics are "computationally difficult to incorporate into training and rarely alleviate all of the harms perpetrated by these systems," suggesting that relying solely on fairness measurements is an incomplete approach to responsible AI.

Explainability can create false confidence: Explanations provided by AI systems can be "gamed to appear fair" and may increase user trust in predictions "even when the explanations are wrong," meaning transparency features don't automatically make systems more trustworthy or accurate.

Explanations pose privacy risks: The paper identifies that explainability and interpretability methods "may inadvertently reduce the privacy of personal information contained in training data," revealing an overlooked tradeoff between transparency and data protection.

Information security practices offer solutions: The authors propose adopting established security methods—including threat modeling, design review, penetration testing, and incident response—to address gaps in current responsible AI approaches and better protect users from potential harms.

Current responsible AI frameworks are incomplete: Despite widespread focus on fairness, explainability, and interpretability as "three pillars" of ethical AI, these approaches "can be challenging to operationalize" in real-world settings, indicating that the field needs more comprehensive development practices.