Feature-Guided Neighbor Selection for Non-Expert Evaluation of Model Predictions

Non-experts can better catch AI errors with the right explanations: In a study of 98 participants without domain expertise, those who received Feature-Guided Neighbor Selection (FGNS) explanations were significantly better at identifying when an AI model made mistakes on Kannada script classification tasks compared to those given traditional explanations.

Speed and accuracy both improved with better-designed explanations: Participants using FGNS made faster and more accurate decisions about model predictions, suggesting that how we present AI reasoning to non-technical users directly impacts their ability to evaluate AI systems responsibly.

Showing representative examples works better than just finding similar cases: FGNS selected example images that better represented the actual characteristics of each script category, rather than simply picking the mathematically closest matches—demonstrating that thoughtful example selection matters more than raw similarity metrics.

Explainability alone doesn't automatically build appropriate trust: Despite improving non-experts' ability to identify errors, the paper notes that "further work is needed to address the gap between explanation quality and perceived trust," highlighting that clear explanations don't automatically make people trust AI systems appropriately.

Interpretability tools should be designed with non-experts in mind: The research demonstrates that XAI methods need to be specifically tested with non-technical users to ensure they actually work in practice, rather than assuming technical explanations will naturally translate to human understanding.