Leaky ReLUs That Differ in Forward and Backward Pass Facilitate Activation Maximization in Deep Neural Networks

Activation maximization, a key explainability tool, has fundamental limitations: The paper demonstrates that AM fails to generate truly optimal input stimuli for neural networks containing ReLUs or Leaky ReLUs, suggesting that visualizations produced by this method may not accurately represent what features networks actually learn to recognize.

Current visualization methods may be misleading for AI interpretation: Because AM struggles to find optimal stimuli, the visual images it generates to explain what neural networks "see" could be unreliable, potentially giving false confidence in our understanding of how these AI systems make decisions.

A technical fix (ProxyGrad) improves activation maximization results: The researchers propose using different activation functions during forward and backward passes of neural networks, which significantly improves the quality of optimal stimuli found by AM and could lead to more trustworthy AI explanations.

The solution has practical benefits beyond explainability: The ProxyGrad approach not only improves activation maximization but also shows promise for training neural networks for image classification tasks, suggesting the technique has applications beyond just making AI systems more interpretable.

This work highlights the importance of scrutinizing explainability methods: The findings underscore that explainable AI techniques themselves need rigorous validation—we cannot assume that popular interpretation methods are working as intended without careful testing.