AHA!: Facilitating AI Impact Assessment by Generating Examples of Harms

AI harms are difficult to predict, but systematic approaches can help: The paper addresses a critical gap in responsible AI practice—that anticipating downstream harms before deployment remains challenging. AHA! provides a structured framework to help practitioners and decision-makers think through potential negative consequences before systems go live.

Different AI problems create different types of harms: By analyzing 4,113 examples of potential harms across five deployment scenarios, the research found that specific problematic AI behaviors (like bias or lack of transparency) tend to produce distinct categories of harms, suggesting that harm prevention strategies should be tailored to the type of AI malfunction.

Combining human and AI perspectives generates more comprehensive harm identification: The study found that using both crowd workers and large language models together to generate harm examples produced more diverse results than either approach alone, suggesting that responsible AI assessment benefits from multiple perspectives rather than relying on a single method.

Responsible AI professionals find systematic harm anticipation valuable: In interviews with nine AI ethics professionals, participants confirmed that AHA!'s structured approach to identifying potential harms was important for ethical reflection and helped them recognize stakeholders and risks they might otherwise have missed.

Harm assessment should consider multiple stakeholders: The framework's emphasis on examining how different groups might be affected by AI failures highlights that responsible AI requires thinking beyond single use cases to understand ripple effects across diverse populations.