Do Concept Bottleneck Models Learn as Intended?

Concept bottleneck models don't work as advertised: These AI systems are designed to make decisions by first identifying human-understandable concepts (like "has stripes" or "is furry" in animal classification), then using those concepts to make predictions. However, the paper finds that the concepts the models learn don't actually correspond to meaningful visual features, undermining their core purpose.

Interpretability claims are questionable: One major selling point of concept bottleneck models is that they're supposed to be more interpretable—meaning humans can understand why the AI made a decision. The researchers' analysis shows this isn't happening; the internal concepts are not semantically meaningful, so users cannot actually understand the model's reasoning.

The gap between theory and practice is significant: While concept bottleneck models were motivated by three important goals—interpretability, predictability, and intervenability (allowing humans to correct the model)—the paper demonstrates these models struggle to achieve any of these in practice, suggesting a fundamental mismatch between how they're designed and how they actually function.

Current approaches to responsible AI may need rethinking: This finding is important for responsible AI development because it suggests that simply adding a "concept layer" to neural networks isn't sufficient to create trustworthy, understandable AI systems. Better approaches may be needed to genuinely achieve interpretability and human control.