What is Missing? Explaining Neurons Activated by Absent Concepts

Neural networks often make decisions based on what's NOT there: The paper reveals that deep learning models frequently activate neurons strongly when certain concepts are absent from an image, not just when they're present. This is a blind spot in current explainability methods that typically only look for what's actively present in the input.

Standard AI explanation tools miss half the story: Popular explainability techniques like attribution methods and feature visualization were designed to identify what features a model uses to make decisions, but they systematically fail to reveal when a model is relying on the absence of features. This means current explanations of AI decision-making are incomplete.

ImageNet models exploit missing concepts in biased ways: The researchers found that computer vision models trained on ImageNet (a widely-used benchmark) exploit encoded absences, suggesting these models may be learning shortcuts or biases related to what's missing from images rather than what's present—potentially explaining some model failures.

Simple fixes can reveal hidden decision-making patterns: The paper proposes straightforward extensions to existing explanation methods that can uncover when models rely on absent concepts, making it possible to audit AI systems more thoroughly for unexpected or problematic decision patterns.

Better explanations could improve AI fairness: By understanding how models use both presence and absence of concepts, researchers can better identify and reduce bias in AI systems, potentially leading to more equitable and trustworthy AI applications.