How Well Do Feature-Additive Explainers Explain Feature-Additive Predictors?

Popular explanation tools have significant limitations: The study tested five widely-used explainers (LIME, SHAP, SHAPR, MAPLE, and PDP) and found that "all explainers eventually fail to correctly attribute the importance of features," meaning they can give misleading explanations even in cases where the underlying model's logic is fully understood.

Feature interactions are a major blind spot: The research reveals that explainers struggle particularly when decisions depend on how multiple features work together rather than independently. This is concerning because real-world decision-making often involves such interactions—for example, a loan approval might depend on both income AND credit score together, not just each factor alone.

The problem persists across different model types: The findings held true whether testing symbolic expressions, neural networks, or generalized additive models across "thousands of synthetic and several real-world tasks," suggesting this is a fundamental issue rather than a problem limited to specific AI systems.

We need better ways to evaluate explanations: The paper highlights that "explanation evaluation remains one of the most challenging issues in XAI," meaning the field currently lacks reliable methods to verify whether an explanation is actually accurate—a critical gap for high-stakes applications like healthcare or criminal justice.

Current explainers may create false confidence: Since these tools are widely deployed to make AI systems seem more trustworthy, their failures could lead decision-makers to trust explanations that are actually incorrect, potentially causing harm in sensitive domains.