Evaluating explainable AI for deep learning-based network intrusion detection system alert classification

Explainability is essential for cybersecurity AI: The study demonstrates that when AI systems are used to filter thousands of daily security alerts, analysts need to understand why the system flagged certain threats as high-priority. Without this transparency, organizations may not trust automated systems to make critical security decisions.

Different explanation methods produce different results: The researchers tested four different techniques for explaining AI decisions (LIME, SHAP, Integrated Gradients, and DeepLIFT) and found significant performance differences. DeepLIFT was most reliable, suggesting that organizations cannot assume all explanation methods are equally trustworthy—the choice of method matters.

Real-world validation with domain experts is crucial: The study involved actual cybersecurity analysts from Tallinn University's Security Operations Center, who helped identify which features (data characteristics) were truly important for alert classification. This collaboration ensured the AI explanations aligned with real security expertise rather than just theoretical metrics.

Explainability requires multiple quality measures: Rather than relying on a single metric, the researchers evaluated explanations across four dimensions: faithfulness (accuracy of explanations), complexity (simplicity), robustness (consistency), and reliability. This comprehensive approach provides a model for assessing whether AI explanations are genuinely trustworthy in high-stakes applications.