ExplainGuard: A Zero Trust Framework for Post-Hoc Explanation Integrity Guarantees in Blackbox XAI Models

XAI explanations can be manipulated by bad actors: The paper demonstrates that adversarial auditors can attack popular explanation methods like SHAP and LIME through techniques such as "output shuffling" and "scaffolding out-of-distribution" to hide model biases while maintaining high accuracy—a practice the authors call "fairwashing."

Current auditing assumes too much trust: The traditional approach to auditing AI systems relies on an implicit assumption that third-party auditors are trustworthy. This paper shows this assumption is flawed and proposes replacing it with a "verify-then-trust" approach instead.

ExplainGuard provides three-layer verification for explanations: The framework checks explanations through three mechanisms before they reach users: detecting if the model has been secretly replaced, verifying explanations are mathematically valid, and confirming that the features highlighted as important are actually stable and faithful to how the model works.

Integrity checks can be done efficiently: The verification system, particularly the feature faithfulness check, is designed with "minimal computational overhead," suggesting these safeguards can be implemented in real-world systems without significant performance costs.

High-stakes AI deployment needs stronger safeguards: As ML models are increasingly used in regulated, high-stakes environments where explanations are required for compliance and building trust, this framework addresses a critical gap in ensuring those explanations are actually reliable and haven't been manipulated.