Evaluating LLM Usage for Efficient and Explainable Numerical and Classified Implicit Sentiment Analysis of Product Desirability

LLMs can effectively measure hidden sentiment in customer feedback: The study found that large language models achieved up to 97% correlation with expert human judgments when analyzing implicit sentiment in product reviews, suggesting they can reliably quantify customer satisfaction without relying on explicit star ratings or review scores.

Smaller, cheaper models can match larger ones' performance: GPT-4o-mini achieved 94% classification accuracy while costing 94% less than larger models, demonstrating that organizations don't need the most expensive AI systems to get reliable results—an important consideration for responsible, cost-conscious AI deployment.

Explainability features build trust in AI decisions: The framework includes human-readable explanations for why the model assigned particular sentiment scores and confidence ratings, which the authors note improves transparency and trust—critical for business decisions that affect product development and customer relations.

LLMs outperformed traditional AI methods significantly: Lexicon-based approaches and standard transformer models "did not produce statistically significant results," while LLMs remained robust across different data formats, suggesting newer AI approaches may be more reliable for this type of nuanced analysis.

Transparency about model confidence matters: By having LLMs express confidence levels alongside their sentiment assessments, organizations can identify cases where the AI is uncertain and may need human review, supporting more responsible decision-making rather than blind reliance on AI outputs.