ExplainabilitySecuritySafetycs.CR
Explainability Guided Adversarial Evasion Attacks on Malware Detectors
Kshitiz Aryal, Maanak Gupta, Mahmoud Abdelsalam, Moustafa Saleh
May 2, 2024
International Conference on Computer Communications and Networks
12 citations
Key Takeaways
-
Explainability tools can be weaponized to make AI systems less secure: The paper demonstrates that explainability techniques like SHAP, which are often promoted as tools for responsible AI, can paradoxically be used to identify exactly where to attack malware detectors. By revealing which parts of files most influence detection decisions, attackers can craft more efficient evasion attacks.
-
AI security requires defending against informed adversaries: The research shows that adversaries with knowledge of how a malware detector works can craft perturbations that fool the system more effectively than random attacks. This highlights that responsible AI deployment must assume attackers will eventually understand model decision-making processes.
-
There's a tension between transparency and security in AI systems: The paper illustrates a fundamental challenge in responsible AI: making systems more interpretable and explainable (generally considered good for accountability) can simultaneously make them more vulnerable to sophisticated attacks if not properly defended.
-
Malware detection systems need multi-layered defenses beyond accuracy metrics: The findings suggest that organizations relying on machine learning for security cannot simply trust high accuracy rates. They need additional safeguards that account for the possibility that adversaries will use explainability information to craft targeted attacks against their systems.
Abstract
As the focus on security of Artificial Intelligence (AI) is becoming paramount, research on crafting and inserting optimal adversarial perturbations has become increasingly critical. In the malware domain, this adversarial sample generation relies heavily on the accuracy and placement of crafted perturbation with the goal of evading a trained classifier. This work focuses on applying explainability techniques to enhance the adversarial evasion attack on a machine-learning-based Windows PE malware detector. The explainable tool identifies the regions of PE malware files that have the most significant impact on the decision-making process of a given malware detector, and therefore, the same regions can be leveraged to inject the adversarial perturbation for maximum efficiency. Profiling all the PE malware file regions based on their impact on the malware detector's decision enables the derivation of an efficient strategy for identifying the optimal location for perturbation injection. The strategy should incorporate the region's significance in influencing the malware detector's decision and the sensitivity of the PE malware file's integrity towards modifying that region. To assess the utility of explainable AI in crafting an adversarial sample of Windows PE malware, we utilize the DeepExplainer module of SHAP for determining the contribution of each region of PE malware to its detection by a CNN-based malware detector, MalConv. Furthermore, we analyzed the significance of SHAP values at a more granular level by subdividing each section of Windows PE into small subsections. We then performed an adversarial evasion attack on the subsections based on the corresponding SHAP values of the byte sequences.
Paper Info
arXiv ID2405.01728
PublishedMay 2, 2024
Citations
12
Relevance72%
VenueInternational Conference on Computer Communications and Networks
TypeJournalArticle, Conference
Categorycs.CR
Responsible AI Topics
ExplainabilitySecuritySafety
Links
Citation data from Semantic Scholar