ExplainabilityTransparencyAccountabilitycs.CL
How good is my story? Towards quantitative metrics for evaluating LLM-generated XAI narratives
Timour Ichmoukhamedov, James Hinns, David Martens
December 13, 2024
arXiv.org
11 citations
1 influential
Key Takeaways
-
LLMs are increasingly used to translate complex AI explanations into plain language, converting technical metrics like SHAP values into user-friendly narratives that help non-experts understand why AI systems make specific decisions.
-
Current evaluation methods rely heavily on human surveys, which are expensive and time-consuming, making the development of automated metrics essential for scaling responsible AI practices and enabling faster iteration on explanation quality.
-
The paper identifies "hallucinations" as a significant problem in LLM-generated explanations, where language models generate plausible-sounding but inaccurate narratives—a critical issue when explanations are meant to build trust in AI systems.
-
Automated metrics can help catch quality problems without human involvement, allowing organizations to systematically evaluate whether their AI explanations are accurate and reliable before deploying them to users.
-
Different LLMs and prompt strategies produce varying quality explanations, suggesting that responsible AI practitioners need systematic evaluation tools to choose the best approaches for generating trustworthy explanations of their AI systems' decisions.
Abstract
A rapidly developing application of LLMs in XAI is to convert quantitative explanations such as SHAP into user-friendly narratives to explain the decisions made by smaller prediction models. Evaluating the narratives without relying on human preference studies or surveys is becoming increasingly important in this field. In this work we propose a framework and explore several automated metrics to evaluate LLM-generated narratives for explanations of tabular classification tasks. We apply our approach to compare several state-of-the-art LLMs across different datasets and prompt types. As a demonstration of their utility, these metrics allow us to identify new challenges related to LLM hallucinations for XAI narratives.