Concept activation vectors: a unifying view and adversarial attacks

CAVs are vulnerable to manipulation through non-concept selection: The paper reveals that Concept Activation Vectors—tools used to explain AI decisions—can be significantly altered by changing which "non-concept" examples are used as comparison points. This means explanations could be misleading if someone deliberately chooses non-representative examples, raising concerns about the reliability of this explanation method.

The choice of comparison data matters more than previously recognized: Prior work on CAVs largely overlooked how sensitive these explanations are to which negative examples are selected. The authors demonstrate this vulnerability through adversarial attacks, suggesting that practitioners need to be much more careful and systematic about this methodological choice.

A probabilistic framework reveals hidden weaknesses: By treating CAVs as random vectors with statistical properties (mean and covariance), the researchers provide a unified theoretical understanding that exposes why the non-concept distribution is so influential—knowledge that can help developers build more robust explanation systems in the future.

Explainability tools require adversarial testing: This work demonstrates that even well-intentioned explanation methods need to be stress-tested against adversarial manipulation, similar to how we test AI systems for other vulnerabilities. Without such testing, organizations might deploy explanations that appear trustworthy but can be subtly compromised.