ExplainabilityTransparencyTrustworthinesscs.CL
Explaining Large Language Models with gSMILE
Zeinab Dehghani, Mohammed Naveed Akram, Koorosh Aslansefat, Adil Khan, Yiannis Papadopoulos
May 27, 2025
4 citations
Key Takeaways
-
We now have better tools to see inside "black box" AI: gSMILE provides a practical method to identify which parts of your input text most influence an AI's output, creating visual heatmaps that show an AI's reasoning path—addressing a major barrier to trusting AI systems in high-stakes decisions like healthcare or legal applications.
-
Different AI models explain themselves differently: The research tested three leading LLMs (GPT-3.5, LLaMA 3.1, and Claude 2.1) and found they vary in how well they can be explained—Claude 2.1 was best at showing attention patterns while GPT-3.5 was most consistent—suggesting that model choice matters when interpretability is important.
-
This approach works across different AI systems: gSMILE is "model-agnostic," meaning it can be applied to various LLMs without needing to redesign it for each one, making it a scalable solution as new AI systems are developed rather than requiring custom explanations for each model.
-
Explainability and performance don't have to conflict: The research demonstrates that gSMILE can help make AI systems more transparent and trustworthy while maintaining their strong performance, suggesting that accountability and capability aren't necessarily trade-offs in AI development.
Abstract
Large Language Models (LLMs) such as GPT, LLaMA, and Claude achieve remarkable performance in text generation but remain opaque in their decision-making processes, limiting trust and accountability in high-stakes applications. We present gSMILE (generative SMILE), a model-agnostic, perturbation-based framework for token-level interpretability in LLMs. Extending the SMILE methodology, gSMILE uses controlled prompt perturbations, Wasserstein distance metrics, and weighted linear surrogates to identify input tokens with the most significant impact on the output. This process enables the generation of intuitive heatmaps that visually highlight influential tokens and reasoning paths. We evaluate gSMILE across leading LLMs (OpenAI's gpt-3.5-turbo-instruct, Meta's LLaMA 3.1 Instruct Turbo, and Anthropic's Claude 2.1) using attribution fidelity, attribution consistency, attribution stability, attribution faithfulness, and attribution accuracy as metrics. Results show that gSMILE delivers reliable human-aligned attributions, with Claude 2.1 excelling in attention fidelity and GPT-3.5 achieving the highest output consistency. These findings demonstrate gSMILE's ability to balance model performance and interpretability, enabling more transparent and trustworthy AI systems.