Should Machine Learning Models Report to Us When They Are Clueless?

AI models frequently make predictions on unfamiliar data without warning users. The paper finds that machine learning models regularly extrapolate beyond their training data—venturing into territory they were never trained on—yet fail to alert stakeholders when this occurs, creating a hidden reliability problem.

Extrapolation detection should be a standard component of AI explainability. While the research community has embraced the importance of explaining AI decisions, the paper argues that knowing when a model is operating outside its area of expertise is equally critical for transparency and should be treated as a fundamental requirement alongside other explainability measures.

Current AI systems lack accountability mechanisms for out-of-distribution predictions. The paper identifies a significant gap: there are no standard practices requiring AI models to report when they encounter unfamiliar samples, meaning users and stakeholders remain unaware when predictions may be unreliable.

Regulatory frameworks should mandate extrapolation reporting. The authors propose practical recommendations for inclusion in AI regulations like the U.S. National AI Initiative Act and the EU AI Act, suggesting that legal requirements for models to flag when they're operating outside their training domain could improve responsible AI deployment.

Transparency about model limitations is essential for accountability. By requiring AI systems to communicate their uncertainty and boundaries, organizations can better protect users and stakeholders from unreliable predictions and make more informed decisions about when and how to deploy AI systems.