DeepSeer: Interactive RNN Explanation and Debugging via State Abstraction

RNNs are difficult to understand and debug: The paper identifies that Recurrent Neural Networks, which are commonly used for language processing tasks, suffer from inherent complexity and lack of transparency, making it hard to know why they make specific predictions or errors.

Most existing explanation tools only show local insights: Current explainable AI techniques for RNNs typically explain individual predictions rather than revealing how the overall model works, limiting users' ability to understand systemic patterns and behaviors.

State abstraction can simplify RNN interpretation: DeepSeer's core innovation bundles similar internal states together and represents the model as a finite state machine, making the model's decision-making process more comprehensible by reducing complexity into interpretable patterns.

Interactive visualization improves debugging effectiveness: In a user study of 28 participants, people using DeepSeer identified root causes of model errors more accurately and developed more actionable improvement plans compared to users of LIME, a popular existing explanation technique, suggesting that interactive tools with multiple coordinated views help practitioners better understand and fix AI systems.

Both global and local explanations are necessary for responsible AI: The paper demonstrates that practitioners need to see both the big picture (how the model generally behaves) and specific details (why particular predictions were made) to effectively assess model reliability and make informed decisions about deployment.