ExplainabilityTrustworthinessTransparencycs.AI
Disproving XAI Myths with Formal Methods -- Initial Results
Joao Marques-Silva
May 13, 2023
IEEE International Conference on Engineering of Complex Computer Systems
19 citations
Key Takeaways
-
XAI tools can create false confidence: The paper identifies "critical misconceptions" in how explainability tools are used, suggesting that popular XAI methods may actually foster distrust rather than build it—meaning organizations deploying these tools could be giving stakeholders misleading confidence in AI systems.
-
Formal methods provide rigorous testing: The authors demonstrate that formal methods (mathematical verification techniques) can be used to disprove common XAI myths, offering a more reliable way to validate whether explanation methods actually work as intended rather than relying on intuition alone.
-
Current XAI approaches have fundamental flaws: The paper indicates that existing explainability techniques contain misconceptions significant enough to warrant formal investigation, suggesting that current industry practices for explaining AI decisions may not be as trustworthy as commonly assumed.
-
Better alternatives exist: Rather than just identifying problems, the authors show how formal methods can help "devise practically effective alternatives," indicating that more rigorous approaches to AI explainability are achievable and implementable.
-
Trust in AI requires scrutiny of trust-building tools: For high-risk domains like healthcare or autonomous systems, the paper emphasizes that we cannot simply assume popular XAI tools are building appropriate trust—they themselves need formal verification to ensure they're actually improving safety and reliability.