Explaining the Unexplainable: A Systematic Review of Explainable AI in Finance

Post-hoc explanations dominate but may miss the full picture: The paper finds that financial institutions heavily rely on post-hoc interpretability techniques (explanations added after a model makes decisions) rather than building transparency into models from the start. This approach may obscure important aspects of how AI systems actually work.

A few techniques carry most of the explanatory load: Attention mechanisms, feature importance analysis, and SHAP are the most commonly used explainability methods in finance. This concentration on a limited toolkit suggests the field may benefit from exploring diverse approaches to ensure robust and comprehensive explanations.

Finance needs domain expertise, not just AI expertise: The review emphasizes that effective explainable AI in finance requires multidisciplinary teams combining financial knowledge with AI explainability skills. This finding suggests that purely technical solutions without financial context may fall short of meaningful transparency.

Current XAI systems have significant gaps: The paper identifies "important shortcomings in present XAI systems" without fully resolving the tension between accuracy and transparency. This indicates that existing explainability methods may not adequately address the real-world needs of financial institutions and their stakeholders.

The accuracy-transparency tradeoff remains unresolved: The abstract notes practitioners are "trying to strike a balance between accuracy and transparency," suggesting this fundamental challenge in AI deployment persists despite advances in explainability techniques.