Contrastive Clustering: Toward Unsupervised Bias Reduction for Emotion and Sentiment Classification

AI classifiers can produce misleading public health insights: The study found that emotion and sentiment classifiers used to analyze social media during crises like COVID-19 can contain biases that skew results, potentially leading public health researchers to draw incorrect conclusions about population sentiment and emotional states.

A new debiasing technique shows promise: Contrastive clustering, an unsupervised method that distinguishes between words that actually indicate emotion versus words that merely correlate with certain topics, achieved an F1 score of 0.753 in identifying bias-prone language—suggesting it could be a practical tool for improving classifier fairness.

Debiasing improves accuracy where it matters most: While the overall performance of classifiers decreased slightly after debiasing (0.02-0.033 points), accuracy improved substantially for the sentences most prone to bias—improving by 0.155 points for anger detection and 0.103 points for negative sentiment—indicating the method targets problems effectively.

Bias reduction produces meaningful changes in results: After debiasing, anger estimates from social media decreased by 14.4% and negative sentiment estimates by 8.0% on average, demonstrating that algorithmic bias can substantially distort the picture of public emotion that researchers and policymakers rely on.

Automated debiasing could enable faster crisis response: By automatically reducing bias without requiring manual annotation, this approach could help public health teams produce more reliable insights more quickly during emergencies when timely information is critical.