ExplainabilityTransparencyAccountabilitycs.HC
Towards User-Focused Research in Training Data Attribution for Human-Centered Explainable AI
Elisa Nguyen, Johannes Bertram, Evgenii Kortukov, Jean Y. Song, Seong Joon Oh
September 25, 2024
arXiv.org
3 citations
Key Takeaways
-
User needs should drive XAI research design: The paper found that many explainable AI practices prioritize mathematical rigor over what practitioners actually need, highlighting a gap between academic solutions and real-world application. The authors argue that involving users early—as they did through interviews with ML developers (N=6) and interactive studies (N=31)—can prevent AI research from becoming impractical.
-
Training data attribution is still malleable and worth shaping now: Because training data attribution (TDA), which explains which training data influenced model decisions, is an emerging field, there's a critical window to build it around human needs rather than technical convenience. The paper warns that without intentional user-centered design, TDA risks repeating mistakes already made in other XAI subfields.
-
Developers need explanations for practical tasks, not just model transparency: Rather than generic explanations, the study revealed that ML developers need tools for specific workflows, such as grouping training samples that cause particular model behaviors or identifying gaps in training data (undersampled categories). This suggests responsible AI tools should be designed around concrete developer problems.
-
Collaboration across disciplines strengthens responsible AI: The paper calls for machine learning researchers, XAI specialists, and human-computer interaction (HCI) experts to work together on these tasks. This interdisciplinary approach can help ensure AI systems are both technically sound and genuinely useful to the people building and maintaining them.