Discovering Concepts in Learned Representations using Statistical Inference and Interactive Visualization

Concept discovery requires human-in-the-loop approaches: The paper finds that fully automated concept discovery and purely manual hand-crafting both have limitations. Instead, semi-automated methods that combine statistical testing and interactive visualization offer a "promising strategy" for finding meaningful concepts when users lack predefined descriptions—suggesting that responsible AI interpretation benefits from human oversight rather than complete automation.

High-dimensional representation spaces are difficult to navigate: The authors identify that neural network representation spaces are inherently "high-dimensional and hard to navigate," which creates a practical barrier for non-experts trying to understand what concepts a model has learned. This highlights why interpretability tools are necessary for making AI systems accessible to stakeholders beyond deep learning specialists.

Statistical testing can guide concept identification: The paper proposes using multiple hypothesis testing as a method to systematically identify which concepts significantly influence a model's classification decisions for specific classes of interest, offering a more rigorous alternative to manual exploration or unsupervised clustering alone.

Interactive visualization enables meaningful concept discovery: The authors demonstrate that interactive visual interfaces can help users explore and validate concepts in learned representations, suggesting that combining computational methods with human judgment through visualization tools makes concept discovery more practical and trustworthy.

Bridging the interpretability gap requires multiple techniques: The research emphasizes that no single approach solves concept discovery, and that combining statistical inference with interactive tools addresses the broader goal of making deep learning models understandable to non-experts and end-users.