ExplainabilityTransparencyAccountabilitycs.CV
Deep Visual City Recognition Visualization
Xiangwei Shi, Seyran Khademi, Jan van Gemert
May 6, 2019
CVPR Workshops
5 citations
Key Takeaways
-
Neural networks learn interpretable city features: The study found that CNNs trained to recognize cities focus on specific visual regions that can be identified and understood, suggesting these models learn meaningful architectural and environmental patterns rather than arbitrary features.
-
Network design matters more than training randomness: The research demonstrated that the choice of neural network architecture has a greater impact on how interpretable the learned features are compared to different random starting points during training, indicating that architectural decisions should be a priority when building explainable AI systems.
-
Visual explanations can be systematically analyzed: Using techniques like Grad-CAM combined with semantic segmentation, researchers were able to quantitatively measure which objects (buildings, streets, etc.) the network uses to distinguish between cities, providing a concrete method for auditing what AI models actually "see."
-
Interpretability requires multiple analytical approaches: The paper employed both unsupervised clustering and supervised semantic labeling to understand CNN decisions, suggesting that no single explanation method is sufficient—responsible AI evaluation benefits from triangulating insights across different analytical techniques.
Abstract
Understanding how cities visually differ from each others is interesting for planners, residents, and historians. We investigate the interpretation of deep features learned by convolutional neural networks (CNNs) for city recognition. Given a trained city recognition network, we first generate weighted masks using the known Grad-CAM technique and to select the most discriminate regions in the image. Since the image classification label is the city name, it contains no information of objects that are class-discriminate, we investigate the interpretability of deep representations with two methods. (i) Unsupervised method is used to cluster the objects appearing in the visual explanations. (ii) A pretrained semantic segmentation model is used to label objects in pixel level, and then we introduce statistical measures to quantitatively evaluate the interpretability of discriminate objects. The influence of network architectures and random initializations in training, is studied on the interpretability of CNN features for city recognition. The results suggest that network architectures would affect the interpretability of learned visual representations greater than different initializations.