Testing the effectiveness of saliency-based explainability in NLP using randomized survey-based experiments

Explanations alone don't guarantee understanding: The study found that saliency-based explanations (which highlight important words in text) don't automatically help humans understand how NLP models work, suggesting that simply making AI "transparent" isn't sufficient for building genuine trust.

Humans are too trusting of explanations: A key finding showed that "humans have a tendency to accept explanations with a less critical view," meaning people may uncritically believe an explanation is correct without properly evaluating whether the AI model's reasoning is actually sound.

Explainability methods need human-centered testing: The research demonstrates that explanation methods must be tested with actual humans through experiments, not just evaluated mathematically, because human psychology and cognitive biases significantly affect how people interpret AI explanations.

High stakes require extra caution: Given that NLP systems are increasingly used in sensitive applications like political profiling and educational essay grading, the study highlights that we cannot assume current explanation methods adequately protect stakeholders from biased or incorrect AI decisions.

Current explainability approaches may create false confidence: The findings suggest that deploying saliency-based explanations without understanding their actual effectiveness on humans could paradoxically harm responsible AI by making people overconfident in flawed model decisions.