SHAP-based Explanations are Sensitive to Feature Representation

How you represent data can change what explanations tell you: Simple choices like whether to represent age as a raw number, a histogram, or age ranges can significantly alter SHAP's assessment of which features are important—even when using the exact same underlying model and data.

These changes can hide discrimination: Because feature representation affects explanations, someone could intentionally (or unintentionally) encode sensitive attributes like race in ways that make discriminatory patterns appear less important in SHAP explanations, potentially obscuring bias in AI systems.

This is different from previously known attacks: While researchers have shown that explanations can be manipulated by poisoning training data or changing models, this paper demonstrates that standard, routine data engineering choices—the kind data scientists make every day—can have the same misleading effect.

"Interpretable" features aren't automatically trustworthy: The paper challenges the assumption that feature-based explanations are reliable just because they use human-understandable features. The way those features are formatted matters enormously for what the explanation reveals.

Responsible AI deployment needs to account for this: Organizations using SHAP or similar explanation tools for high-stakes decisions (hiring, lending, criminal justice) should be aware that their data preprocessing choices could be inadvertently—or deliberately—distorting what these explanations reveal about model behavior.