VARSHAP: Addressing Global Dependency Problems in Explainable AI with Variance-Based Local Feature Attribution

VARSHAP addresses a real problem with current explainability tools: Popular methods like SHAP can give misleading explanations because they rely on global data patterns rather than focusing on what actually matters for individual predictions, meaning explanations may not reflect how a model truly behaves in specific situations.

The new method uses variance reduction as its core principle: VARSHAP determines feature importance by measuring how much a feature reduces uncertainty in predictions, which is a more intuitive way to identify which features genuinely drive individual decisions compared to existing approaches.

VARSHAP maintains mathematical rigor while improving practical performance: The method satisfies important mathematical properties (Shapley axioms) that ensure fair and consistent explanations, while also being more robust when data distributions change—a common real-world challenge that SHAP struggles with.

Performance improvements are demonstrated across multiple scenarios: Experiments show VARSHAP outperforms both KernelSHAP and LIME on both synthetic test cases and real-world datasets, suggesting the improvements are practical and not limited to artificial conditions.

The method works across different types of AI models: As a model-agnostic approach, VARSHAP can be applied to any machine learning model without modification, making it potentially useful for explaining diverse AI systems in practice.