ExplainabilityTransparencyFairnesscs.CL
HaT5: Hate Language Identification using Text-to-Text Transfer Transformer
Sana Sabah Sabry, Tosin Adewumi, Nosheen Abid, György Kovacs, Foteini Liwicki, Marcus Liwicki
February 11, 2022
IEEE International Joint Conference on Neural Network
20 citations
Key Takeaways
-
AI models can nearly match human-level performance on hate speech detection: The T5 model achieved macro F1 scores of 81.66% and 82.54% on two major hate speech datasets (OLID 2019 and HASOC 2021), coming very close to state-of-the-art results of 82.9% and 83.05%, demonstrating the technical feasibility of automated content moderation systems.
-
Explainability is critical for building user trust in AI systems: The researchers used Integrated Gradient analysis to explain why their models made specific predictions, emphasizing that "explainable artificial intelligence (XAI) is essential for earning the trust of users"—a key principle for responsible deployment of content moderation tools.
-
Data quality problems undermine AI system reliability: The paper revealed significant annotation errors in the HASOC 2021 dataset, finding cases where the model made correct predictions that contradicted the labeled "ground truth," highlighting how poor data annotation can compromise the trustworthiness of AI systems trained on that data.
-
Transparency and reproducibility support responsible AI development: The authors published their model checkpoints on HuggingFace to "foster transparency," making their work reproducible and allowing other researchers to audit and build upon their approach rather than treating the system as a "black box."
-
Data augmentation can improve hate speech detection performance: Using an autoregressive conversational AI model to augment training data improved model performance, suggesting that strategic data enhancement techniques can help build more effective content moderation systems.