Towards Reconciling Usability and Usefulness of Explainable AI Methodologies

Explanations that feel understandable aren't always the most accurate: The study found a disconnect between what users perceive as usable and what actually helps them understand AI decisions correctly. Participants rated text-based explanations as more usable than other formats, but this perception didn't necessarily translate to better comprehension of how the AI actually worked.

One-size-fits-all explanation methods don't work: The research tested four different explanation approaches (Decision Trees, Text, Programs, and others) for explaining self-driving car decisions and found that different formats have different strengths and weaknesses. This suggests developers need to consider who will use the system and tailor explanations accordingly, rather than applying a single explanation method universally.

Technical transparency isn't enough for real-world accountability: The paper emphasizes that simply making AI systems "transparent" by revealing their internal mechanics doesn't solve the problem—end-users often lack the technical knowledge to understand these explanations in a reasonable timeframe. This gap between technical transparency and actual user understanding creates accountability risks when AI systems make harmful decisions.

Designing explainable AI requires understanding human perception: To create truly useful explanations, developers must study how different groups of people actually perceive and process AI explanations, not just assume that clearer technical information will automatically lead to better understanding.