The benefits and costs of explainable artificial intelligence in visual quality control: Evidence from fault detection performance and eye movements

XAI speeds up decisions but doesn't always improve accuracy: While explainable AI highlighting reduced decision time in visual inspection tasks, the impact on error rates "were highly dependent on (X)AI accuracy," meaning faster decisions don't guarantee better ones.

Misleading explanations can be worse than no explanation: When XAI highlights pointed to the wrong area (marking intact parts while missing actual faults), participants made more errors. This "cost" of XAI occurred because people relied on the highlights rather than conducting thorough independent analysis.

XAI can reduce human effort and attention: Eye-tracking data showed that when given XAI highlights, participants "spent less time searching the rest of the mould and thus looked at the fault less often," suggesting people may become passive rather than critically engaged when AI provides visual guidance.

System accuracy matters critically for XAI effectiveness: The benefits of explainable AI only appeared "when the system correctly detected and highlighted the fault," indicating that XAI is only trustworthy when the underlying AI system is reliable—a significant practical limitation.

Individual differences affect how people use XAI: The researchers observed "large interindividual differences" in how participants responded to XAI support, suggesting that the same explanation tool may help some workers while misleading others, with important implications for deployment across teams.