Rashomon in the Streets: Explanation Ambiguity in Scene Understanding

Multiple accurate models can give conflicting explanations: The study found significant disagreement in explanations between different models that all correctly predict the same driving scenarios, demonstrating that there is no single "correct" explanation for why an AI system makes a particular decision in autonomous driving contexts.

The problem is fundamental, not fixable through better modeling: The researchers concluded that explanation ambiguity appears to be an inherent property of the task itself rather than a flaw that can be eliminated by improving model design, which has important implications for how much we should rely on AI explanations in safety-critical applications.

Even simple, interpretable models show explanation disagreement: The study compared two very different model types—straightforward gradient boosting models and complex Graph Neural Networks—and found substantial disagreement between them, suggesting the issue affects both transparent and "black box" AI systems.

Current explanation methods may mask uncertainty: The findings raise concerns about feature attribution methods (techniques used to explain which factors influenced an AI decision), suggesting these methods may present explanations with false confidence even when the underlying explanation is genuinely ambiguous.

Autonomous driving validation faces a hidden challenge: For safety-critical applications like self-driving cars, this research reveals that validating AI systems through their explanations may be less reliable than previously assumed, since multiple conflicting explanations could all be technically valid.