The Impact of Concept Explanations and Interventions on Human-Machine Collaboration

Concept Bottleneck Models improve transparency but don't automatically boost performance: While CBMs successfully made AI decision-making more interpretable compared to standard neural networks, this increased understanding did not translate into significantly better task accuracy when humans and machines worked together—challenging earlier lab-based findings that only tested models without human involvement.

Understanding AI requires sustained engagement, not one-time explanations: The study found that humans needed multiple interactions to grasp how the model made decisions, suggesting that effective AI transparency is an ongoing process rather than a one-time explanation, which has implications for how organizations design AI interfaces.

Better alignment between humans and AI doesn't guarantee better outcomes: Even when humans better understood the model's reasoning and became more aligned with it, this didn't reliably improve their collaborative performance on tasks, indicating that interpretability alone is insufficient for effective human-AI teamwork.

Real-world human evaluation reveals gaps in AI research: This was the first human study of Concept Bottleneck Models in collaborative settings, and it revealed that findings from controlled lab experiments—where researchers manually corrected model errors—don't necessarily hold when actual humans interact with the system, highlighting the importance of testing AI systems with real users before deployment.

Misalignment between human and machine reasoning persists despite transparency efforts: The abstract notes ongoing misalignment between how models and humans make decisions, suggesting that making AI more explainable is necessary but not sufficient for true human-machine collaboration.