Data Descriptions from Large Language Models with Influence Estimation

Language-based explanations can improve AI transparency: The paper demonstrates that generating textual descriptions of data using large language models helps humans understand what information AI systems are actually learning from, moving beyond traditional "black-box" prediction explanations to explain the data itself.

Not all AI-generated explanations are equally useful: The researchers found that influence estimation—a technique for identifying which descriptions are most informative—significantly improves the quality of data explanations, suggesting that filtering out irrelevant information is crucial for responsible AI systems that communicate with users.

Better data descriptions can enhance model performance: The study shows that models trained only on images performed better across nine image classification datasets when supplemented with high-quality textual descriptions, indicating that understanding data more thoroughly can lead to more reliable AI systems.

Cross-modal AI evaluation provides practical validation: By testing whether descriptions generated for one type of data (images) could improve performance in another domain, the researchers demonstrated a rigorous method for validating that AI explanations are genuinely capturing meaningful information rather than just sounding plausible.

This approach supports more interpretable AI decision-making: The paper suggests that by understanding how data is represented and described within AI systems, stakeholders can gain insights into why models make specific decisions, which is essential for building trust and accountability in AI applications.