ExplainabilityTransparencyTrustworthinesscs.CV
Locality Guided Neural Networks for Explainable Artificial Intelligence
Randy Tan, Naimul Khan, Ling Guan
July 12, 2020
IEEE International Joint Conference on Neural Network
9 citations
1 influential
Key Takeaways
-
Neurons can be organized for better human understanding: The paper demonstrates that by clustering neighboring neurons based on their correlations, researchers can make deep neural networks more interpretable—showing how groups of neurons interact rather than treating hundreds of independent neurons as a black box.
-
Locality Guided Neural Networks (LGNN) improves explainability without sacrificing performance: The proposed method enforces a local topology where neighboring neurons are highly correlated, achieving this transparency "without changing the structure of current models nor requiring any post processing," while maintaining competitive classification accuracy on CIFAR100 image recognition tasks.
-
This approach addresses a fundamental XAI challenge: By organizing neurons spatially within each layer, the method tackles the core problem that makes deep learning opaque to humans—the difficulty of understanding how hundreds of independent neurons in deeper layers interact with one another.
-
The solution is broadly applicable across AI architectures: While the experiments focus on CNNs (VGG and Wide ResNet), the authors note the method "can theoretically be applied to any type of deep learning architecture," suggesting potential for improving explainability across different AI systems.
-
Explainability doesn't require architectural redesign: Unlike some XAI approaches that require post-processing or model restructuring, LGNN achieves interpretability through a modified training algorithm, making it potentially easier to integrate into existing AI development workflows.