Information That Matters: Exploring Information Needs of People Affected by Algorithmic Decisions

People affected by AI decisions need different information than technical experts: The study found that AI novices—including domain experts and those directly impacted by algorithmic decisions—have distinct information needs that current AI explanations typically fail to address, covering categories like data quality, system context, how the system is used, and technical specifications.

Explanations alone don't guarantee understanding: While participants reported increased confidence after receiving explanations about employment prediction and health monitoring systems, they still faced significant challenges in locating relevant information and accurately assessing whether they actually understood how the systems worked.

People's risk perception shapes what they want to know: The research revealed that participants who perceived high risks from an AI system prioritized learning about the intentions behind deploying it, while those with lower risk perceptions asked different types of questions—suggesting that one-size-fits-all explanations may not serve diverse stakeholder needs.

There's a gap between what AI systems explain and what matters to affected people: The study highlights a critical disconnect: explanations of AI systems rarely address the specific information needs of those personally impacted by algorithmic decisions, particularly AI novices who lack technical background.

People struggle to verify their own understanding: Participants had difficulty assessing whether they truly comprehended how systems worked and sometimes attempted to "outsource" understanding rather than develop it themselves, suggesting the need for better tools to help people evaluate their own comprehension.