LLMs for XAI: Future Directions for Explaining Explanations

LLMs can make AI explanations more accessible: The paper demonstrates that Large Language Models can transform technical ML explanations into natural, human-readable narratives, potentially making AI decision-making more understandable to non-experts who lack technical backgrounds.

User studies show promise but more work is needed: Initial user study results suggest LLMs improve the interpretability and usability of explanations, but the paper identifies several open research questions—including how to properly evaluate these enhanced explanations and design effective prompts—indicating this is still an emerging approach.

The focus is on improving existing explanations, not replacing them: Rather than using LLMs to directly explain AI models, the researchers refine explanations that already come from established XAI algorithms, suggesting a complementary approach that builds on existing explainability methods rather than replacing them entirely.

Multiple design choices affect explanation quality: The paper highlights that prompt design, choice of LLM model, and integration of external data all influence how well LLMs can transform explanations, meaning organizations implementing this approach will need to carefully consider these technical decisions.

Standardized evaluation metrics are still lacking: The paper identifies defining appropriate evaluation metrics as a key research direction, suggesting that the field currently lacks clear standards for measuring whether LLM-generated explanations actually improve human understanding of AI decisions.