Causal effect of racial bias in data and machine learning algorithms on user persuasiveness & discriminatory decision making: An Empirical Study

Biased training data directly causes discriminatory AI outcomes: The study demonstrates a causal link between racial bias present in language datasets and discriminatory decisions made by AI/NLP models, showing this is not merely a correlation but a direct effect of how models learn from biased data.

Biased AI systems undermine user trust and decision-making: When users interact with AI systems trained on racially biased data, they become less confident in the system's explanations and are more likely to make poor decisions based on the model's outputs, indicating that bias doesn't just affect the algorithm—it affects human judgment downstream.

Poor model explainability amplifies societal harms: The research found that biased AI models produce unexplainable discriminatory outcomes, meaning users cannot understand why the system made a particular decision, which compounds the ethical problem by making it harder to identify and correct unfair treatment.

Bias in AI systems magnifies existing societal inequities: Rather than remaining neutral, AI models trained on biased language data actively reinforce and amplify ethnic, religious, gender, and socioeconomic biases already present in society, raising serious concerns about AI's role in perpetuating discrimination at scale.

Responsible AI requires attention to data quality from the start: The findings underscore that building fair and trustworthy AI systems requires careful examination and mitigation of bias in training datasets before models are deployed, as downstream harms to users and society are difficult to reverse.