Evaluating Standard Feature Sets Towards Increased Generalisability and Explainability of ML-based Network Intrusion Detection

Standard feature sets don't automatically transfer across different networks: The paper found that while ML-based intrusion detection systems achieve near-perfect accuracy in controlled lab settings, their performance varies significantly when applied to different real-world network environments and attack types—highlighting a critical gap between research claims and practical deployment.

NetFlow features outperform alternatives for detecting diverse attacks: Among the feature sets tested (NetFlow and CICFlowMeter), NetFlow demonstrated superior detection accuracy across three different datasets (CSE-CIC-IDS2018, BoT-IoT, and ToN-IoT), suggesting that standardizing on this feature set could improve real-world security system performance.

Explainability is essential for trustworthy AI security systems: The researchers used SHAP (SHapley Additive exPlanations) to make ML model decisions interpretable, analyzing which features actually influence security predictions. This transparency is crucial for organizations to understand why their AI system flagged a threat and to trust its recommendations.

Not all features contribute equally to security decisions: By examining Shapley values across datasets, the paper revealed that different features have varying importance depending on the network context, meaning one-size-fits-all feature selection may not be appropriate for all organizational environments.

Closing the research-to-practice gap requires rigorous evaluation: The paper addresses why excellent academic results haven't translated to widespread deployment—comprehensive evaluation across multiple real-world scenarios and explainability analysis are necessary before deploying ML security systems in production.