Explaining the Road Not Taken

Users want contrastive explanations, but most AI interpretability research doesn't provide them. The paper found that people frequently ask "why did the model choose option A instead of option B?" but the dominant explanation methods (feature attribution, decision rules, and probes) cannot effectively answer these comparative questions.

There's a significant gap between what researchers study and what users actually need. By comparing over 200 NLP papers against the XAI Question Bank, the authors identified a mismatch: the field focuses on explaining individual model decisions rather than addressing users' real questions about why the model rejected plausible alternatives.

"The road not taken" matters for trust and decision-making. Users care about understanding not just what a model decided, but why it didn't choose seemingly similar, legitimate alternatives—a distinction that current explanation methods largely ignore, potentially limiting how effectively people can evaluate and trust AI systems.

Current explanation approaches may be insufficient for responsible AI deployment. If existing interpretability methods cannot answer the questions users actually ask, this raises concerns about whether current explainability efforts genuinely support human oversight and accountability in real-world AI applications.