What Emotions Make One or Five Stars? Understanding Ratings of Online Product Reviews by Sentiment Analysis and XAI

Machine learning models can appear accurate while hiding underlying biases: The paper found that algorithms like XGBoost and Random Forests showed good performance metrics, but explainable AI (XAI) methods revealed they were making predictions through flawed mechanisms—demonstrating that high accuracy scores alone don't guarantee trustworthy AI systems.

Dataset imbalance is a critical but often-overlooked problem: Study 3 revealed a 64.4% "no-information rate" due to class imbalance in the review data, meaning the dataset was heavily skewed toward certain ratings. This fundamental data problem undermined model reliability despite initial performance metrics looking acceptable.

Explainable AI methods are essential for catching prediction bias: The paper used XAI visualization techniques (local feature attributions and partial dependency plots) to uncover incorrect prediction patterns that wouldn't have been visible from performance metrics alone, showing that transparency tools are necessary for responsible AI deployment.

Emotional language patterns alone may not reliably predict ratings: While "joy" and "negative valence" were identified as the most predictive sentiment features, the paper's findings suggest these emotional indicators don't capture the full complexity of why people give certain ratings, highlighting the limitations of oversimplified feature selection.

Responsible AI requires scrutinizing data quality before deployment: The paper concludes that practitioners must examine whether their datasets are biased toward certain predictions before trusting model outputs in real-world applications like consumer recommendation systems.