BiasFairnessAccountabilitycs.IR
Bias in Conversational Search: The Double-Edged Sword of the Personalized Knowledge Graph
Emma J. Gerritse, Faegheh Hasibi, Arjen P. de Vries
October 20, 2020
International Conference on the Theory of Information Retrieval
30 citations
2 influential
Key Takeaways
-
Personalization creates a bias trade-off: While personalized knowledge graphs (PKGs) make conversational AI systems more useful by tailoring answers to individual users, this same personalization mechanism can amplify existing biases and trap users in echo chambers where they only encounter information aligned with their prior beliefs.
-
Multiple sources of bias require different solutions: The paper identifies three distinct types of bias—people bias (user preferences), algorithm bias (system design), and combinations of both—each requiring tailored strategies rather than one-size-fits-all approaches to bias mitigation in conversational search.
-
Bias measurement is critical but complex: The paper emphasizes that evaluating bias in conversational systems requires both objective measurement methods and assessment of user satisfaction, suggesting that technical metrics alone are insufficient for understanding real-world bias impacts.
-
Structural design choices matter: The way conversational systems store and use personal information through PKGs fundamentally shapes whether they will amplify or mitigate bias, making responsible system architecture a key lever for responsible AI deployment in personal devices.
Abstract
Conversational AI systems are being used in personal devices, providing users with highly personalized content. Personalized knowledge graphs (PKGs) are one of the recently proposed methods to store users' information in a structured form and tailor answers to their liking. Personalization, however, is prone to amplifying bias and contributing to the echo-chamber phenomenon. In this paper, we discuss different types of biases in conversational search systems, with the emphasis on the biases that are related to PKGs. We review existing definitions of bias in the literature: people bias, algorithm bias, and a combination of the two, and further propose different strategies for tackling these biases for conversational search systems. Finally, we discuss methods for measuring bias and evaluating user satisfaction.