Study on the Helpfulness of Explainable Artificial Intelligence

One-size-fits-all explanations don't work: The paper finds that different XAI methods perform differently depending on context, emphasizing that there is no single best explanation method—the right approach depends on the specific situation and what users need to understand.

User studies reveal what metrics alone cannot: Testing explanations with actual people uncovers important human factors that technical measurements miss, such as whether users can truly understand the explanation and make better decisions based on it, not just whether the explanation is mathematically sound.

Explanations affect trust in different ways: The user study found that different XAI methods vary in their ability to generate appropriate trust or skepticism in AI systems—some may make users overly confident in wrong decisions while others may create unnecessary doubt in correct ones.

Helpfulness should be measured by real-world decision-making: Rather than evaluating explanations in isolation, the paper proposes testing whether explanations actually help people make better decisions on relevant tasks, providing a more practical measure of whether XAI is truly serving its purpose.

Human-centered evaluation is essential for responsible AI: The authors recommend that developers of AI systems should conduct user studies to test explanations with real people before deployment, especially in critical domains like medicine or autonomous vehicles where poor understanding could have serious consequences.