On Background Bias of Post-Hoc Concept Embeddings in Computer Vision DNNs

Concept-based explainability methods can inherit background biases from training data: The paper demonstrates that popular post-hoc concept explanation techniques (Net2Vec-based methods) frequently capture spurious correlations between concepts and backgrounds rather than learning the concepts themselves. For example, wild animals are typically associated with vegetation backgrounds, and these methods may exploit this shortcut instead of truly understanding what makes an animal a "wild animal."

Background bias can cause dangerous performance failures in real-world scenarios: The research identifies "alarming" performance degradation when concepts appear in atypical backgrounds—such as animals on roads instead of natural settings. These corner cases could go undetected by standard evaluation practices, potentially leading to unreliable AI systems deployed in safety-critical applications.

Current explainability techniques don't adequately account for uncontrolled training backgrounds: While concept-based XAI methods have been useful for uncovering biases in how models treat different target classes, the paper reveals that the methods themselves are vulnerable to background bias because backgrounds are typically uncontrolled during neural network training.

Simple background randomization techniques can improve concept explanations: Testing 3 established background randomization approaches across 7 different neural network architectures and 50+ concepts, the authors show that even low-cost interventions can provide valuable insights and improve how well concept explanations work, suggesting practical solutions are available.

Responsible AI requires scrutinizing the explainability tools themselves: This work highlights that tools designed to make AI more interpretable and trustworthy can themselves introduce hidden biases, emphasizing the need for critical evaluation of explanation methods before relying on them for AI accountability.