Evaluating the Stability of Semantic Concept Representations in CNNs for Robust Explainability

Concept explanations in AI systems need to be stable and reliable: The paper demonstrates that when AI systems explain their decisions using semantic concepts (like "wheels" or "windows"), these explanations must remain consistent and accurate to be trustworthy for safety-critical applications like autonomous driving.

Simpler concept representations aren't always better: The research found that reducing the complexity of how concepts are stored in neural networks can actually improve stability, challenging the assumption that more detailed representations automatically lead to better explanations.

Training data quantity matters for concept stability: The study shows that the number of examples used to teach the AI system about concepts directly impacts how reliably those concepts are retrieved and used for explanations—a practical consideration for building trustworthy AI systems.

Gradient-based explanations can be unreliable: The paper identifies that some popular methods for explaining AI decisions (those based on mathematical gradients) suffer from instability issues, meaning they may produce different explanations for the same decision depending on minor variations in the input.

Current explainability methods need validation before deployment: The findings suggest that before using concept-based explanations for inspecting AI errors or making safety decisions, organizations should actively test whether these explanations remain consistent and reliable across different scenarios.