Comparing Facial Expression Recognition in Humans and Machines: Using CAM, GradCAM, and Extremal Perturbation

Humans significantly outperform machines at recognizing facial expressions, suggesting that despite advances in deep neural networks, current AI systems have not yet matched human-level capability in this task—an important finding for applications like emotion detection or mental health monitoring where accuracy matters.

Different AI explanation methods reveal different attention patterns, with Extremal Perturbation aligning best with how humans actually look at faces during recognition. This highlights that the choice of explainability technique (CAM, GradCAM, or Extremal Perturbation) substantially affects our understanding of what an AI system is "paying attention to."

Comparing AI attention to human attention is crucial for responsible AI development, as this research demonstrates. Understanding whether machines focus on the same facial features as humans helps identify potential biases or blind spots in AI systems before they're deployed in high-stakes applications.

Explainability methods matter for trustworthiness: The finding that some explanation techniques better match human reasoning than others suggests that organizations deploying facial recognition systems should carefully evaluate which explanation methods they use—not all transparency tools are equally informative about how the system actually works.