FovEx: Human-Inspired Explanations for Vision Transformers and Convolutional Neural Networks

FovEx improves explainability across different AI architectures: The method achieves state-of-the-art performance on both Vision Transformers (4 out of 5 metrics) and convolutional neural networks (3 out of 5 metrics), addressing a key limitation of existing explanation techniques that often work well for only specific model types.

Human-inspired explanations are more trustworthy: FovEx's explanations align significantly better with actual human gaze patterns than existing methods, showing a +14% improvement over RISE and +203% improvement over GradCAM in alignment metrics, which suggests these explanations better reflect how humans naturally understand images.

Computational efficiency matters for practical AI deployment: While perturbation-based explanation methods are model-agnostic, they require many forward passes and are computationally expensive; FovEx combines the efficiency of gradient-based methods with the flexibility of perturbation approaches, making explainability more practical for real-world applications.

Biological inspiration can improve AI interpretability: By mimicking human foveal vision—how humans focus on specific regions of interest—FovEx demonstrates that understanding human perception can lead to better AI explanation methods, bridging the gap between how machines and humans process visual information.