Mitigating Explanation Leakage in Financial Fraud Detection Systems

Privacy and transparency create conflicting demands in financial AI: The paper identifies a fundamental tension where regulatory requirements for model explainability (using tools like TreeSHAP) directly conflict with privacy protections in federated learning systems, as detailed explanations can be exploited to infer whether specific customers' data was used in training.

Explanation data itself is a privacy vulnerability: The research demonstrates that high-fidelity SHAP explanations are susceptible to Membership Inference Attacks, meaning bad actors can reverse-engineer which individuals' financial data contributed to a fraud detection model—a risk that extends beyond the model weights themselves.

A new approach protects privacy at the explanation layer: DP-FedSHAP applies differential privacy specifically to explanations rather than to the underlying model, offering a targeted solution that may better preserve both the usefulness of fraud detection and the privacy of individual financial records compared to existing methods.

Trade-offs between competing goals require careful measurement: The study emphasizes that solutions must balance three competing priorities—keeping explanations accurate enough to be useful, protecting individual privacy sufficiently, and maintaining the model's ability to catch fraud (measured by AUPRC)—rather than optimizing any single goal.

Imbalanced real-world data complicates privacy-utility decisions: Testing on the IEEE-CIS Fraud Detection dataset (which reflects the reality that fraud is rare) shows that privacy protections may have different impacts depending on data characteristics, suggesting one-size-fits-all solutions may not work across different financial institutions.