ExplainabilityTransparencySafetycs.LG
Explain to Not Forget: Defending Against Catastrophic Forgetting with XAI
Sami Ede, Serop Baghdadlian, Leander Weber, An Nguyen, Dario Zanca, Wojciech Samek, Sebastian Lapuschkin
May 4, 2022
International Cross-Domain Conference on Machine Learning and Knowledge Extraction
16 citations
Key Takeaways
-
Neural networks forget previous knowledge when learning new tasks: The paper addresses "catastrophic forgetting," where AI systems completely lose previously learned information when trained on new data—a significant limitation compared to how humans continuously learn and retain knowledge.
-
Explainability can improve AI learning efficiency: The researchers use explainability techniques (Layer-wise Relevance Propagation) to identify which parts of a neural network are important for previous tasks, then protect that information when learning new tasks, demonstrating a practical connection between AI transparency and system performance.
-
The proposed method uses fewer computational resources: Unlike existing solutions to catastrophic forgetting, this approach achieves better knowledge retention while requiring less computational power and data, which has implications for making responsible AI more practical and sustainable.
-
This addresses a real barrier to practical AI deployment: The ability to continuously learn and adapt to new information without losing previous capabilities is essential for AI systems deployed in real-world environments, making this research relevant to building more reliable and trustworthy AI systems.
Abstract
The ability to continuously process and retain new information like we do naturally as humans is a feat that is highly sought after when training neural networks. Unfortunately, the traditional optimization algorithms often require large amounts of data available during training time and updates wrt. new data are difficult after the training process has been completed. In fact, when new data or tasks arise, previous progress may be lost as neural networks are prone to catastrophic forgetting. Catastrophic forgetting describes the phenomenon when a neural network completely forgets previous knowledge when given new information. We propose a novel training algorithm called training by explaining in which we leverage Layer-wise Relevance Propagation in order to retain the information a neural network has already learned in previous tasks when training on new data. The method is evaluated on a range of benchmark datasets as well as more complex data. Our method not only successfully retains the knowledge of old tasks within the neural networks but does so more resource-efficiently than other state-of-the-art solutions.