SoK: Privacy Attacks on Machine Learning via Explainable AI

Explanations create new security vulnerabilities: The paper systematizes 25 studies showing that AI explanations—tools designed to make models more transparent—can be exploited to steal models, infer whether specific people's data was used in training, and reconstruct private information from the model, revealing that transparency features have unintended privacy costs.

How attackers access explanations matters as much as the explanations themselves: The researchers identify five distinct paths through which adversaries obtain explanation signals (target-released, attacker-derived, secondary disclosure, privileged access, and released global artifacts), emphasizing that the same explanation poses different risks depending on whether it's publicly available versus obtained through insider access.

No single defense protects against all explanation-based attacks: The analysis finds that different explanation types have different vulnerabilities and that existing defenses are inconsistently effective, meaning organizations cannot rely on one-size-fits-all protection strategies and must instead match defenses to specific attack scenarios and the particular assets they're trying to protect.

Explanation privacy requires holistic risk assessment: Rather than treating explanations as simply "safe" or "unsafe," the paper argues that organizations must evaluate privacy risks as end-to-end problems by considering what signal is exposed, how attackers might acquire it, which data or models are at risk, and what background knowledge adversaries already possess.

Transparency and privacy involve inherent tradeoffs: The findings suggest that deploying explainable AI systems requires explicit decisions about which privacy risks are acceptable in exchange for the benefits of model transparency, rather than assuming that better explanations automatically serve users' interests.