ExplainabilityTransparencyTrustworthinesscs.AI
Explaining How a Neural Network Play the Go Game and Let People Learn
Huilin Zhou, Huijie Tang, Mingjie Li, Hao Zhang, Zhenyu Liu, Quanshi Zhang
October 15, 2023
arXiv.org
10 citations
1 influential
Key Takeaways
-
AI systems can encode strategic knowledge beyond human expertise: The paper demonstrates that neural networks trained to play Go have learned patterns and strategies that surpass human understanding, suggesting AI models may discover novel insights in complex domains that could benefit human learning.
-
Explainability requires mathematical rigor, not intuition: The authors emphasize that extracting knowledge from AI models demands formal mathematical foundations rather than speculative interpretation, ensuring that humans learn "accurate and verifiable knowledge" rather than potentially misleading patterns.
-
Interaction primitives can bridge AI and human understanding: By identifying and extracting fundamental "interaction primitives between stones" from the neural network's value function, the research shows a concrete method for translating opaque AI decision-making into interpretable concepts that humans can actually learn from and verify.
-
Responsible AI development includes knowledge transfer: This work addresses an important aspect of responsible AI—not just building capable systems, but actively designing them to be interpretable and educational, allowing humans to benefit from and validate the knowledge AI systems have learned.
Abstract
The AI model has surpassed human players in the game of Go, and it is widely believed that the AI model has encoded new knowledge about the Go game beyond human players. In this way, explaining the knowledge encoded by the AI model and using it to teach human players represent a promising-yet-challenging issue in explainable AI. To this end, mathematical supports are required to ensure that human players can learn accurate and verifiable knowledge, rather than specious intuitive analysis. Thus, in this paper, we extract interaction primitives between stones encoded by the value network for the Go game, so as to enable people to learn from the value network. Experiments show the effectiveness of our method.