ExplainabilityTransparencyAccountabilitycs.LG
Explaining high-dimensional text classifiers
Odelia Melamed, Rich Caruana
November 22, 2023
arXiv.org
1 citations
Key Takeaways
-
Explainability methods need updating for complex AI systems: Traditional explainability tools have significant limitations when applied to neural networks processing high-dimensional data (like text), suggesting that existing methods may not adequately explain how these widely-used AI systems actually make decisions.
-
New theoretical approaches can improve transparency: The paper demonstrates a novel explainability method grounded in mathematical properties of neural networks, offering a more principled way to understand how these models work compared to ad-hoc explanation techniques.
-
Practical applications span multiple domains: The method was validated on both sentiment analysis (IMDB reviews) and cybersecurity (malware detection in PowerShell scripts), indicating that improved explainability tools can benefit diverse real-world AI applications where understanding decisions matters.
-
Better explanations support responsible AI deployment: By providing clearer insights into how text classifiers reach their conclusions, this work enables stakeholders—including developers, auditors, and end-users—to better identify potential biases, errors, or unexpected behaviors before deployment.
Abstract
Explainability has become a valuable tool in the last few years, helping humans better understand AI-guided decisions. However, the classic explainability tools are sometimes quite limited when considering high-dimensional inputs and neural network classifiers. We present a new explainability method using theoretically proven high-dimensional properties in neural network classifiers. We present two usages of it: 1) On the classical sentiment analysis task for the IMDB reviews dataset, and 2) our Malware-Detection task for our PowerShell scripts dataset.