Symbolic Knowledge Extraction and Injection with Sub-symbolic Predictors: A Systematic Literature Review

Making AI systems more understandable is crucial for trust: The paper addresses a fundamental problem with modern AI systems—they often work like "black boxes" that make predictions without explaining their reasoning. By extracting human-readable rules and knowledge from these systems, organizations can better understand and justify AI decisions to users and stakeholders.

Two complementary approaches can improve AI transparency: The research identifies symbolic knowledge extraction (SKE)—pulling out interpretable rules from trained AI models—and symbolic knowledge injection (SKI)—feeding human knowledge back into AI systems before or during training. Together, these methods help bridge the gap between how AI systems actually work and how humans understand decision-making.

A large toolkit of methods already exists but needs better guidance: The systematic review analyzed 132 SKE methods and 117 SKI methods from existing literature, yet notes that many lack readily available software implementations. This suggests that while researchers have developed numerous techniques, practitioners may struggle to apply them in real-world settings without additional support.

Different AI systems and use cases require different approaches: The paper's taxonomies categorize methods by their purpose, how they operate, what data they need, and what types of AI predictors they work with. This suggests there is no one-size-fits-all solution—organizations need to carefully match the right transparency method to their specific AI system and business needs.

Responsible AI development requires both extracting and injecting knowledge: Rather than treating AI systems as fixed black boxes, the research promotes a cycle where human expertise can be incorporated into AI systems and then extracted again for verification, supporting more accountable and trustworthy AI deployment.