SafetyExplainabilityTrustworthinesscs.AI
Rule-based Out-Of-Distribution Detection
Giacomo De Bernardi, Sara Narteni, Enrico Cambiaso, Maurizio Mongelli
March 3, 2023
IEEE Transactions on Artificial Intelligence
9 citations
Key Takeaways
-
AI systems need safeguards to detect when real-world data differs from training data: The paper addresses a critical deployment challenge where machine learning models may encounter unfamiliar situations in operation that weren't present during training, potentially leading to unsafe autonomous decisions.
-
Explainability can be used as a tool for safety monitoring: Rather than relying on complex statistical assumptions, the researchers developed a method using explainable AI (XAI) techniques to identify when operational data significantly differs from training conditions, making the safety monitoring process more transparent and interpretable.
-
The approach was validated across diverse real-world applications: Testing on predictive maintenance, autonomous vehicle platooning, and cybersecurity scenarios demonstrated that the method can both accurately detect problematic data shifts and assess how different the operational environment is from training conditions.
-
Open-source implementation enables broader adoption: By making their code and data publicly available, the researchers support reproducibility and allow other organizations to implement similar safety checks in their own AI systems without proprietary barriers.
Abstract
Out-of-distribution detection is one of the most critical issue in the deployment of machine learning. The data analyst must assure that data in operation should be compliant with the training phase as well as understand if the environment has changed in a way that autonomous decisions would not be safe anymore. The method of the paper is based on eXplainable Artificial Intelligence (XAI); it takes into account different metrics to identify any resemblance between in-distribution and out of, as seen by the XAI model. The approach is non-parametric and distributional assumption free. The validation over complex scenarios (predictive maintenance, vehicle platooning, covert channels in cybersecurity) corroborates both precision in detection and evaluation of training-operation conditions proximity. Results are available via open source and open data at the following link: https://github.com/giacomo97cnr/Rule-based-ODD.