ExplainabilityTransparencyTrustworthinesscs.HC
User Decision Guidance with Selective Explanation Presentation from Explainable-AI
Yosuke Fukuchi, Seiji Yamada
February 28, 2024
2024 33rd IEEE International Conference on Robot and Human Interactive Communication (ROMAN)
6 citations
Key Takeaways
-
Strategic explanation selection can influence user decisions: The paper demonstrates that Intelligent Decision Support Systems don't need to show all available explanations—instead, selectively presenting certain explanations can effectively guide users toward AI-recommended decisions, suggesting that how explanations are presented matters as much as what is explained.
-
This approach works best when AI is highly accurate: X-Selector improved task performance, but crucially "under the condition of a high AI accuracy," indicating that selective explanation presentation is only beneficial when the underlying AI system is reliable; using this method with unreliable AI could mislead users into following incorrect recommendations.
-
Selective presentation raises transparency concerns: By strategically choosing which explanations to show users, X-Selector prioritizes decision alignment over full transparency—this creates a tension in responsible AI between helping users make better decisions and ensuring they have complete information to make informed choices.
-
Current comparison lacks critical baselines: The study compares X-Selector against showing all explanations or no explanations, but doesn't test against a neutral approach of showing randomly selected explanations, making it unclear whether the method's success comes from strategic selection or simply from reducing cognitive overload.
-
Real-world deployment requires careful consideration: Before implementing selective explanation systems in high-stakes domains (healthcare, finance, criminal justice), organizations should consider whether users have a right to see all available evidence, not just explanations chosen to guide them toward a particular decision.