"Trust Junk" Leads to Unjustified Support for Highly Discriminatory Predictive Models

Visual explanations can create false confidence: The study found that providing accurate but irrelevant data visualizations in model explanations led people to trust discriminatory AI systems they should have rejected, demonstrating that "trust junk" can override rational judgment about fairness.

More information doesn't always mean better decisions: Superfluous data presented visually actually increased unjustified support for biased models, suggesting that explainable AI (XAI) designers should focus on relevant information rather than comprehensive data dumps.

Visualizations have persuasive power beyond their content: The research reveals that the rhetorical design of explanations—how information is presented visually—can manipulate user perception independently of whether a model is actually fair or discriminatory.

Current XAI practices may inadvertently enable bias: By showing that people can be convinced to support patently discriminatory models through visual explanations, the paper warns that well-intentioned transparency efforts could actually help harmful AI systems gain acceptance rather than catch problems.

Responsible AI requires critical design thinking: XAI developers must actively consider how their visualizations might persuade users to trust models uncritically, rather than assuming that providing explanations automatically leads to better human judgment about AI systems.