Unraveling the Real Working Mechanism and Inherent Flaws of GAE: A Method for Interpreting Transformer Processes from an Economic Perspective

Current XAI methods prioritize performance metrics over actual interpretability: The paper argues that explainable AI research has become overly focused on improving scores on proxy metrics rather than ensuring the methods themselves are truly interpretable and rational, creating a gap between what appears to work well and what actually explains AI decisions.

Popular explanation methods like GAE may have fundamental flaws: Generic Attention-model Explainability (GAE), a widely-used technique for interpreting transformer models, has inherent working mechanism issues that the authors identify, suggesting practitioners should reconsider whether this common interpretation approach is actually reliable.

We need "explainability of explainability" (XXAI): The paper advocates for a new research direction focused on understanding and validating XAI methods themselves, rather than just accepting them at face value—essentially arguing that explanation methods deserve the same scrutiny we apply to the AI systems they're meant to explain.

Attention patterns alone don't fully explain transformer behavior: By focusing on attention processes, GAE and similar methods may miss important aspects of how transformers actually work, suggesting that current popular explanation techniques provide an incomplete picture of model decision-making.

Domain applications of GAE-based methods may need re-evaluation: Given the identified flaws, the authors suggest that real-world applications relying on GAE for model interpretation should be re-examined to ensure they're making decisions based on sound explanations rather than flawed interpretations.