AUTOLYCUS: Exploiting Explainable AI (XAI) for Model Extraction Attacks against Interpretable Models

Explainability tools create security vulnerabilities: The paper demonstrates that XAI tools designed to make AI decisions more transparent—specifically LIME and SHAP, which are widely used in industry—can paradoxically make models easier to steal through "model extraction attacks" where attackers create unauthorized copies of proprietary models.

Attackers can reverse-engineer models with fewer attempts: AUTOLYCUS requires "significantly fewer queries compared to state-of-the-art attacks" to successfully copy a model's functionality, meaning bad actors need less interaction with a system to compromise it—a particular concern for companies protecting valuable proprietary models.

There's a fundamental tension between transparency and security: The research highlights a critical trade-off in responsible AI: while explainability is important for accountability and user trust, the explanations themselves can be weaponized by attackers, forcing organizations to choose between being interpretable and being secure.

Black-box protections are insufficient: Even when companies try to protect their models by only allowing black-box access (hiding internal parameters and architecture), the explanations provided to users can leak enough information for attackers to reconstruct the model—suggesting current security practices may be inadequate.

Popular XAI methods need security considerations: Because LIME and SHAP are "extensively adopted, simplicity, and usability" features that make them attractive to practitioners, the security risks identified in this paper affect a broad range of real-world AI systems currently in deployment.