Report on the First Knowledge Graph Reasoning Challenge 2018 -- Toward the eXplainable AI System

Explainability is becoming essential for AI safety: The challenge was specifically designed to address the growing importance of AI systems that can explain their reasoning, not just make predictions. This reflects a recognition that for AI to be safely deployed in society, users need to understand why an AI system reached a particular decision.

Formal methods and rule-based approaches outperformed deep learning for explainability: The winning solution used constraint satisfaction and formal methods rather than deep learning, while second place used SPARQL and rules. This suggests that for tasks requiring transparent reasoning, traditional symbolic AI approaches may be more effective than the deep learning methods that currently dominate AI applications.

Knowledge graphs provide a structured foundation for explainable reasoning: The challenge used an open knowledge graph based on Sherlock Holmes stories as its foundation, demonstrating that organizing information in structured, interconnected formats can enable AI systems to provide traceable, logical explanations for their conclusions.

Multiple complementary approaches exist for explainable AI: The diverse winning submissions—ranging from formal methods to rule-based systems to multi-agent models—suggest there is no single solution to explainability, and different techniques may be appropriate for different types of problems.

Explainable AI development requires collaborative, standardized evaluation: By organizing a formal challenge with shared datasets and evaluation metrics, the researchers created a framework for the AI community to collectively advance explainability techniques, similar to how other challenges have driven progress in AI capabilities.