ChatGPT Rates Natural Language Explanation Quality Like Humans: But on Which Scales?

ChatGPT can evaluate AI explanations similarly to humans, but scale matters: The study found that ChatGPT's ratings of explanation quality align better with human judgments when using simpler rating systems (like binary yes/no or three-point scales) rather than more detailed 7-point scales, suggesting that AI assessment tools work best with straightforward evaluation frameworks.

Fine-grained human evaluation remains challenging and resource-intensive: The research highlights that collecting detailed human ratings for explanation quality is complex and subjective, requiring 900 annotations across just 300 examples—underscoring why automated assessment tools like ChatGPT could be valuable for responsible AI development, though with important limitations.

Specific prompting techniques can improve AI assessment reliability: When researchers provided ChatGPT with semantically similar examples in prompts (called "dynamic prompting"), the alignment with human judgments improved, suggesting that how we ask AI systems to evaluate explanations significantly affects their accuracy and usefulness.

Paired comparisons outperform direct ratings: The study found that asking ChatGPT to compare two explanations side-by-side produced better alignment with human judgment than asking it to rate explanations on absolute scales, offering a practical approach for organizations seeking to automate explanation quality assessment.

Automation of explanation evaluation could support responsible AI, but with caveats: While ChatGPT shows promise in assessing whether AI explanations are clear and informative, the findings suggest it should be used strategically—particularly for initial screening or coarse-grained assessments—rather than as a complete replacement for human oversight in responsible AI practices.