ExplainabilityTrustworthinessTransparencycs.AI
Cybertrust: From Explainable to Actionable and Interpretable AI (AI2)
Stephanie Galaitsi, Benjamin D. Trump, Jeffrey M. Keisler, Igor Linkov, Alexander Kott
January 26, 2022
arXiv.org
1 citations
Key Takeaways
-
Trust requires predictability over perfect explanation: The paper argues that users don't need AI systems to explain every decision in human terms. Instead, trust builds through consistent, predictable behavior over time—meaning reliable performance matters more than detailed explanations of how AI works.
-
Confidence metrics are essential: Rather than forcing AI to justify itself, systems should explicitly show users how confident the AI is in its recommendations. This allows people to know when to rely on AI versus when to seek human judgment, creating a more honest relationship with the technology.
-
Explainability can limit AI capabilities: The paper warns that requiring AI to only make decisions humans can easily understand may unnecessarily constrain what AI can accomplish. The goal should be interpretability (understanding what the system does) rather than explainability (understanding why in human terms).
-
Testing and verification build justified trust: Rather than trusting AI based on explanations, users should be able to examine and test AI predictions to verify they work reliably. This evidence-based approach creates a stronger foundation for trust than narrative explanations alone.
-
Decision-analytic tools matter: Incorporating formal decision-making frameworks and visualization tools into AI systems helps users understand confidence levels and make better choices about when to follow AI recommendations—shifting responsibility appropriately between human and machine judgment.