Using Large Language Models and Knowledge Graphs to Improve the Interpretability of Machine Learning Models in Manufacturing

Making AI decisions understandable matters in real-world settings: The paper demonstrates that in manufacturing environments, workers need to understand why an AI system made a particular decision. By combining knowledge graphs with large language models, the researchers created explanations that were clearer and more useful than traditional AI explanations, which is critical for workers who must act on these recommendations.

Structured knowledge improves AI explanations: Rather than having an AI system explain itself in isolation, the method stores domain-specific manufacturing information in an organized "knowledge graph" that the AI can reference. This grounding in real-world context—similar to how a human expert would reference their training—produced more accurate and consistent explanations across 33 different test questions.

Complex, context-specific questions reveal AI system strengths: The researchers went beyond standard evaluation questions to test more nuanced, real-world scenarios specific to manufacturing. This approach revealed that their method excels at handling the kinds of complicated, multi-layered questions that actual workers would ask, suggesting that responsible AI evaluation should test systems against realistic use cases rather than generic benchmarks.

Transparency and usability are distinct challenges: The paper distinguishes between technical accuracy of explanations and whether workers actually find them clear and useful. Both metrics were evaluated, highlighting that responsible AI requires not just correct answers, but answers presented in ways that non-technical users can understand and trust.