XAI for Transformers: Better Explanations through Conservative Propagation

Gradient-based explanations for AI systems are unreliable: The paper demonstrates that simply using gradient information to explain how Transformers (a widely-used AI architecture) make decisions doesn't accurately show which input features actually matter, because gradients only reflect local behavior rather than the full decision-making process.

Specific architectural components cause explanation problems: Attention Heads and LayerNorm—key components of Transformer models—are identified as the primary culprits behind inaccurate explanations, meaning these specific parts need special attention when trying to understand AI decisions.

A better method for AI transparency is now available: The researchers propose an improved explanation technique called Conservative Propagation, which extends an established method (LRP) to work properly with Transformers and provides more reliable insights into how these AI systems reach their conclusions.

Improved explanations work across different AI applications: The new method was tested on multiple Transformer models and datasets, showing that it consistently outperforms gradient-based approaches, suggesting it could help increase transparency across many real-world AI applications.

Better AI explanations support responsible deployment: By providing more accurate explanations of how Transformers make decisions, this work helps organizations and regulators better understand and audit AI systems before deploying them in high-stakes domains like healthcare, finance, or criminal justice.