Evaluation Metrics for Symbolic Knowledge Extracted from Machine Learning Black Boxes: A Discussion Paper

Rule extraction from AI systems is increasingly important for transparency: As opaque machine learning models are deployed across many application fields, the paper identifies rule extraction techniques as a promising approach to make these "black box" systems more understandable and readable to end-users who need to trust and verify AI decisions.

There is no agreed-upon standard for measuring readability of extracted knowledge: The paper highlights that while techniques exist to extract human-interpretable rules from complex AI models, the field lacks a quantitative metric to assess how readable or understandable these extracted rules actually are—a gap that prevents fair comparison between different extraction methods.

Readability assessment is more complex than it appears: The authors emphasize that evaluating readability involves multiple "puzzling issues" and criticalities that go beyond simple metrics, suggesting that what makes knowledge truly readable to humans varies depending on context, user expertise, and the specific knowledge representation format used.

Standardized readability metrics could enable better AI development: The paper argues that developing agreed-upon evaluation metrics would allow researchers to automatically compare different knowledge extraction approaches and create parameter autotuning algorithms, ultimately leading to more transparent and trustworthy AI systems.

Multiple knowledge representations require different evaluation approaches: The discussion acknowledges that different ways of representing extracted knowledge (such as different rule formats) exist, implying that a one-size-fits-all readability metric may not be feasible.