Feature Importance versus Feature Influence and What It Signifies for Explainable AI

Feature importance and feature influence are fundamentally different concepts: The paper clarifies that feature importance measures how much changing a feature can alter model outcomes, while feature influence (used by most current explainable AI methods) is measured against a reference baseline. This distinction matters because using the wrong metric could lead to misleading explanations of why a model made a particular decision.

Current explainable AI methods may be providing incomplete explanations: Most state-of-the-art post-hoc explanation methods rely on feature influence rather than true feature importance, which means they may not be fully capturing how features actually drive model decisions. This has implications for stakeholders trying to understand and trust AI systems.

The Contextual Importance and Utility (CIU) method offers a more comprehensive approach: CIU provides a unified framework that works for both global explanations (how a model generally works) and local explanations (why a specific decision was made), and includes a "value utility" concept that assesses whether individual feature values are favorable for the outcome—offering more nuanced insights than influence-based methods alone.

Better explanations require measuring both what matters and how much it matters: By combining contextual importance with contextual utility, explanations become "more expressive and flexible," allowing stakeholders to understand not just which features influenced a decision, but whether those feature values were actually beneficial or harmful for the outcome.

Explanation method choice affects trustworthiness and accountability: The paper's assessment of fidelity and stability across different methods suggests that organizations implementing explainable AI should carefully evaluate which explanation methods they use, as different approaches may produce significantly different—and potentially misleading—insights into model behavior.