ExplainabilityTransparencyAccountabilitycs.LG
Streamlining models with explanations in the learning loop
Francesco Lomuscio, Paolo Bajardi, Alan Perotti, Elvio G. Amparore
February 15, 2023
International Conference on Data Science and Advanced Analytics
Key Takeaways
-
Explanations can improve model design: The paper demonstrates that insights from AI explanation methods (which show why a model made a specific decision) can be fed back into the model-building process itself, creating more efficient models that are easier to understand.
-
Simpler models can match complex ones: By using explanation information during training, researchers created "streamlined models" that are more compact than original black-box models while maintaining similar accuracy levels—suggesting that transparency and performance don't have to be traded off against each other.
-
Better alignment between explanations and actual reasoning: The new approach helps ensure that the explanations shown to users actually reflect how the model is making decisions, rather than explanations being added after the fact to a model that may reason differently than the explanations suggest.
-
Two practical strategies for implementation: The paper proposes specific, actionable methods (Iterative Dataset Weighting and Targeted Replacement Values) that practitioners can use to incorporate explanation feedback into their model development workflows.
-
Potential for more trustworthy AI systems: By creating models whose internal reasoning better matches their explanations to users, this work supports the development of AI systems that are more honest and easier to audit for potential biases or errors.