Do you see what I see? An Ambiguous Optical Illusion Dataset exposing limitations of Explainable AI

AI systems rely on subtle visual cues that may not align with human reasoning: The research found that machine learning models heavily weight specific features like gaze direction and eye cues when interpreting ambiguous images, yet these same features don't necessarily drive human perception in the same way, revealing a fundamental misalignment between how AI "sees" and how humans see.

Explainable AI tools have blind spots when facing ambiguity: By testing models with optical illusions—images intentionally designed to be perceptually ambiguous—the researchers exposed limitations in current explainability methods, suggesting that these tools may give false confidence about model decision-making in genuinely uncertain situations.

Ambiguous data is common in real-world applications but understudied: The paper highlights that optical illusions are rare in existing datasets despite their relevance to safety-critical domains like autonomous driving and medical diagnostics, where real-world data is often ambiguous and models must handle uncertainty responsibly.

Visual bias in AI systems deserves systematic investigation: The researchers created a structured dataset with controlled variations to study how different visual concepts influence model behavior, providing a foundation for identifying and potentially mitigating hidden biases in computer vision systems before deployment.

Testing AI with deliberately challenging cases improves safety: By confronting models with perceptual ambiguity rather than only testing on clear-cut examples, the research demonstrates the value of adversarial evaluation approaches for uncovering model vulnerabilities in high-stakes applications.