ExplainabilitySafetyTransparencycs.CV
Improving Explainability of Disentangled Representations using Multipath-Attribution Mappings
Lukas Klein, João B. S. Carvalho, Mennatallah El-Assady, Paolo Penna, Joachim M. Buhmann, Paul F. Jaeger
June 15, 2023
International Conference on Medical Imaging with Deep Learning
5 citations
Key Takeaways
-
Disentangled representations go beyond pixel-level explanations: Traditional explainability methods only highlight which pixels matter for a diagnosis, but this paper shows that breaking down a model's internal representations into interpretable components allows experts to understand why certain visual features are being used—a crucial distinction for clinical safety.
-
The framework enables causal reasoning in medical AI: By visualizing these disentangled representations, domain experts can leverage their medical knowledge to investigate potential causal relationships rather than just correlations, addressing a key gap in current explainable AI approaches for high-stakes healthcare decisions.
-
Multi-path attribution mapping validates explanations: The paper introduces a validation technique that enriches explanations through multiple pathways, helping clinicians and regulators gain confidence that the model's reasoning is sound and not based on spurious patterns.
-
Shortcut detection improves model robustness without extra testing: The framework can identify when models rely on problematic "shortcuts" (like using irrelevant features) without requiring expensive testing under different data distributions—a practical advantage for deploying safer clinical AI systems.
-
Effectiveness demonstrated on real medical data: The approach was validated on synthetic benchmarks and two actual medical datasets, suggesting the framework has practical applicability beyond theoretical proof-of-concept for clinical diagnostic systems.