TRUST-LAPSE: An Explainable and Actionable Mistrust Scoring Framework for Model Monitoring

ML models need continuous monitoring to know when their predictions are trustworthy: The paper demonstrates that even trained models can fail in unpredictable ways, making it essential to have systems that flag when a model's predictions should not be relied upon—particularly important for high-stakes applications like medical seizure detection.

The TRUST-LAPSE framework uses mathematical distance measurements to detect when models encounter unfamiliar data: By measuring how far new inputs deviate from patterns the model learned during training (using Mahalanobis distance and cosine similarity), the system can identify when a model is operating outside its area of expertise, achieving detection rates of 84.1% accuracy in vision tasks and 77.1% in clinical EEG data.

Existing monitoring approaches have critical blind spots that make them unreliable: The research reveals that popular baseline methods fail to detect when input data has fundamentally changed in meaningful ways, making them "unfit for real-world model monitoring"—a finding that challenges current industry practices.

The framework is designed to be actionable and explainable for practitioners: Unlike black-box monitoring systems, TRUST-LAPSE provides clear signals about why a prediction should be questioned and when to take action (such as requesting human review), making it practical for real-world deployment rather than just a research tool.

The approach works across different types of data and real medical applications: Testing on audio, images, and actual patient EEG recordings for seizure detection demonstrates the framework's versatility and readiness for deployment in critical healthcare settings.