Visualization Of Class Activation Maps To Explain AI Classification Of Network Packet Captures

Explainability is essential for building trust in AI systems: The paper emphasizes that deep learning models used for network traffic classification function as "black-box models, which weakens the experts' trust in the classifications," highlighting that high performance alone is insufficient—users need to understand why the AI made its decisions.

Visualization techniques can bridge the gap between AI and human experts: By combining classification with explanation techniques (specifically Class Activation Maps), the researchers created an interface that allows network security experts to interpret AI decisions rather than blindly accepting them, demonstrating that explainability tools can make AI systems more practical for real-world deployment.

Explainability enables model improvement and knowledge discovery: Beyond trust, the paper notes that explanations allow experts to evaluate models, "gain new insights from the data and improve the model"—suggesting that understanding AI decisions can lead to better systems and reveal patterns humans might otherwise miss.

Complex AI systems require human-centered design: The interactive visual tool described represents a responsible AI approach by recognizing that experts need accessible interfaces to work effectively with AI, rather than assuming that technical performance metrics alone justify deployment in critical infrastructure like network security.