Evaluating The Explainability of State-of-the-Art Deep Learning-based Network Intrusion Detection Systems

Deep learning security systems lack transparency: State-of-the-art deep learning models used to detect network intrusions operate as "black boxes," making it difficult to understand how they decide whether network traffic is malicious or safe—a critical concern when these systems protect critical infrastructure.

Explainability tools often contradict each other: The research found that different explanation methods produce conflicting results for most intrusion detection models tested, meaning experts cannot reliably agree on why a system flagged a particular threat, undermining confidence in the system's decisions.

Some AI security models are more explainable than others: The study demonstrates that explainability varies significantly across different deep learning architectures, suggesting that organizations have choices when selecting intrusion detection systems and should prioritize models that can better justify their decisions.

There's a trade-off between accuracy and explainability: The researchers identified tension between building highly accurate threat detection systems and building ones that can explain their reasoning, requiring security teams to make deliberate choices about which priority matters more for their specific needs.

Standardized evaluation criteria are needed: The paper introduces new methods to measure whether explanations are consistent and reliable, highlighting that the field lacks agreed-upon standards for assessing whether AI security systems are truly trustworthy and understandable.