ExplainabilityAccountabilityTransparencycs.HC
Exploring Practitioner Perspectives On Training Data Attribution Explanations
Elisa Nguyen, Evgenii Kortukov, Jean Y. Song, Seong Joon Oh
October 31, 2023
arXiv.org
2 citations
Key Takeaways
-
Training data quality is practitioners' top priority: The study found that model developers consider training data quality "the most important factor for high model performance in practice," yet they rely primarily on personal experience rather than systematic approaches to curate data—suggesting a gap between what matters most and how it's currently managed.
-
Training data attribution explanations are largely unknown and unused: Despite being a focus of XAI research, the interviewed practitioners were unfamiliar with training data attribution (TDA) techniques and did not use them, indicating a significant disconnect between academic research and real-world adoption.
-
End-users want explanations that improve their interaction with AI, not just technical details: The study found that end-users prioritize explanations that enhance how they work with models rather than necessarily wanting to understand training data specifically, suggesting that explanation design should focus on practical utility.
-
There's a need to evaluate TDA techniques against real-world use cases: The researchers call for the AI community to shift focus "from the human-machine collaboration perspective" and evaluate training data attribution methods based on how they actually help practitioners solve problems, rather than purely technical metrics.
-
Explainable AI requires understanding practitioner needs: The interdisciplinary nature of XAI means that technical solutions must be designed with input from the people who will actually use them, not developed in isolation from practical workflows and constraints.