ExplainabilityTransparencyAccountabilitycs.CV
Beyond the Black Box: Do More Complex Deep Learning Models Provide Superior XAI Explanations?
Mateusz Cedro, Marcin Chlebus
May 14, 2024
arXiv.org
1 citations
Mateusz Cedro, Marcin Chlebus
Simpler AI models can outperform complex ones: In this study of lung X-ray analysis, the simplest model (ResNet-18) achieved 98.4% accuracy compared to 95.9% for the most complex model (ResNet-101), challenging the assumption that "bigger is better" in AI development.
Model complexity doesn't improve explainability: Across 11 out of 12 statistical tests, more complex models did not produce meaningfully better explanations of their decisions. This suggests that adding complexity to AI systems may not help us understand why they make specific predictions—a critical concern for healthcare applications.
Explainability metrics showed no significant differences: Both the "Relevance Rank Accuracy" and "Positive Attribution Ratio" metrics revealed that simpler and complex models provided similarly relevant explanations, indicating that explanation quality is independent of model complexity in this healthcare context.
Implications for responsible AI deployment: These findings suggest that organizations implementing AI in healthcare should prioritize simpler, more interpretable models that maintain high accuracy rather than automatically adopting the most sophisticated available options, potentially reducing risks and improving trustworthiness.
Trade-offs matter in high-stakes domains: The research demonstrates that in sensitive applications like medical imaging, the pursuit of marginal performance gains through increased complexity may come at the cost of reduced accuracy and no corresponding benefit to explainability.
The increasing complexity of Artificial Intelligence models poses challenges to interpretability, particularly in the healthcare sector. This study investigates the impact of deep learning model complexity and Explainable AI (XAI) efficacy, utilizing four ResNet architectures (ResNet-18, 34, 50, 101). Through methodical experimentation on 4,369 lung X-ray images of COVID-19-infected and healthy patients, the research evaluates models' classification performance and the relevance of corresponding XAI explanations with respect to the ground-truth disease masks. Results indicate that the increase in model complexity is associated with a decrease in classification accuracy and AUC-ROC scores (ResNet-18: 98.4%, 0.997; ResNet-101: 95.9%, 0.988). Notably, in eleven out of twelve statistical tests performed, no statistically significant differences occurred between XAI quantitative metrics - Relevance Rank Accuracy and the proposed Positive Attribution Ratio - across trained models. These results suggest that increased model complexity does not consistently lead to higher performance or relevance of explanations for models' decision-making processes.
15 pages, 9 figures, 5 tables
Citation data from Semantic Scholar