A Systematic Literature Review on Explainability for Machine/Deep Learning-based Software Engineering Research

Black-box AI models in software engineering pose real risks: The paper identifies that AI/ML models used for critical SE tasks like vulnerability detection lack transparency in their decision-making, creating safety concerns that currently prevent these otherwise promising tools from being deployed in practice.

Explainability techniques are being applied across 23 different software engineering tasks: The systematic review of 108 papers shows that explainability methods are being actively researched and implemented across diverse SE applications, indicating growing recognition of the need for transparency in this domain.

There is no standardized approach to evaluating explainability: The paper identifies that existing studies lack consistent evaluation methods for assessing whether explainability techniques actually work, suggesting the field needs to establish clearer standards for measuring transparency and trustworthiness.

The gap between research and real-world deployment remains significant: Despite advances in explainability research, the paper emphasizes that AI-driven SE models are "still far from being deployed in practice," indicating that understanding why an AI makes decisions is necessary but not yet sufficient for practical adoption.

Guidelines and best practices for future work are needed: The authors highlight specific challenges and opportunities for improvement, suggesting that responsible deployment of AI in software engineering requires not just better explanations, but also clearer frameworks for when and how to use these tools safely.