ExplainabilityTransparencyTrustworthinesseess.SP
Evaluation of Interpretability for Deep Learning algorithms in EEG Emotion Recognition: A case study in Autism
Juan Manuel Mayor-Torres, Sara Medina-DeVilliers, Tessa Clarkson, Matthew D. Lerner, Giuseppe Riccardi
November 25, 2021
Artif. Intell. Medicine
66 citations
4 influential
Key Takeaways
-
Existing AI explanation methods are unreliable when features are statistically connected: The paper demonstrates that current popular explainability techniques (like Layer-Wise Relevance Propagation and Grad-based methods) lack reliability when the data features used to train AI models are entangled or correlated with each other—a common real-world scenario that undermines trust in clinical AI applications.
-
Deep learning is being rapidly deployed in autism diagnosis without adequate transparency safeguards: While AI models are increasingly used in clinical trials to predict neurodevelopmental disorders like Autism Spectrum Disorder, the paper highlights that "inclusion of more reliable saliency-maps...is still insufficiently mature for practical applications in diagnostics or clinical trials," raising concerns about premature clinical deployment.
-
A new validation method (ROAR) can better identify which brain signals actually matter: The researchers propose the RemOve-And-Retrain (ROAR) methodology as a more trustworthy alternative to existing explanation techniques, offering a practical way to verify which features an AI model genuinely relies on rather than just appears to rely on.
-
Interpretability validation is essential before clinical use: This study emphasizes that before AI systems analyzing brain activity (EEG) are used to diagnose conditions in vulnerable populations like autistic individuals, researchers must rigorously verify that the AI's explanations are actually accurate and not misleading.