Concept Induction using LLMs: a user experiment for assessment

LLMs can generate more understandable AI explanations than traditional methods: The study found that concepts generated by GPT-4 were significantly more comprehensible to humans than those produced by ECII, a traditional concept induction system, suggesting LLMs offer a practical improvement over existing automated explanation approaches.

Human explanations still outperform AI-generated ones: Despite GPT-4's strong performance, human-generated concepts remained superior in terms of understandability, indicating that while LLMs are valuable tools for explainability, they have not yet fully replicated human reasoning and communication abilities.

Minimal data requirements make this approach practical: The method uses only "minimal textual object information" available in datasets through prompting, meaning organizations don't need extensive additional annotation or resources to generate explanations—a significant advantage for real-world deployment of explainable AI systems.

Human evaluation is essential for assessing explanation quality: The researchers conducted a user study because "there is no established metric to determine the human understandability of concepts," highlighting that technical metrics alone cannot validate whether AI explanations actually make sense to people—a critical consideration for responsible AI development.

This addresses a real gap in explainable AI: The work tackles a genuine challenge in XAI: traditional post-hoc explanation methods "often struggle to deliver interpretable explanations," and this LLM-based approach offers a more accessible alternative for making complex image classification models transparent to users.