AI2030 Responsible AI Tools Marketplace

Key Takeaways

LLMs inherit and amplify societal biases from their training data, generating stereotypical content related to gender, race, profession, and religion—a critical concern as these models become increasingly integrated into applications that affect real people.
The researchers created a substantial new dataset of 52,751 labeled examples of stereotypical text to enable better detection and auditing of bias in language models, addressing a gap in tools available for responsible AI evaluation.
Multi-class classification approaches outperformed simpler binary methods for detecting stereotypes, suggesting that more sophisticated detection frameworks may be necessary to catch the nuanced ways bias appears in AI-generated text.
Analysis of GPT models over time showed a reduction in stereotypical bias, indicating that developers can make measurable progress on this issue—though the paper implies ongoing vigilance and auditing remain necessary.
The framework provides a practical tool for auditing LLMs, enabling organizations and researchers to systematically evaluate whether their language models are generating biased content before deployment in real-world applications.

Abstract

Large Language Models (LLM) have made significant advances in the recent past becoming more mainstream in Artificial Intelligence (AI) enabled human-facing applications. However, LLMs often generate stereotypical output inherited from historical data, amplifying societal biases and raising ethical concerns. This work introduces i) the Multi-Grain Stereotype Dataset, which includes 52,751 instances of gender, race, profession and religion stereotypic text and ii) a novel stereotype classifier for English text. We design several experiments to rigorously test the proposed model trained on the novel dataset. Our experiments show that training the model in a multi-class setting can outperform the one-vs-all binary counterpart. Consistent feature importance signals from different eXplainable AI tools demonstrate that the new model exploits relevant text features. We utilise the newly created model to assess the stereotypic behaviour of the popular GPT family of models and observe the reduction of bias over time. In summary, our work establishes a robust and practical framework for auditing and evaluating the stereotypic bias in LLM.

2023 NeurIPS SoLaR Workshop Accepted

Key Takeaways

LLMs inherit and amplify societal biases from their training data, generating stereotypical content related to gender, race, profession, and religion—a critical concern as these models become increasingly integrated into applications that affect real people.
The researchers created a substantial new dataset of 52,751 labeled examples of stereotypical text to enable better detection and auditing of bias in language models, addressing a gap in tools available for responsible AI evaluation.
Multi-class classification approaches outperformed simpler binary methods for detecting stereotypes, suggesting that more sophisticated detection frameworks may be necessary to catch the nuanced ways bias appears in AI-generated text.
Analysis of GPT models over time showed a reduction in stereotypical bias, indicating that developers can make measurable progress on this issue—though the paper implies ongoing vigilance and auditing remain necessary.
The framework provides a practical tool for auditing LLMs, enabling organizations and researchers to systematically evaluate whether their language models are generating biased content before deployment in real-world applications.

Abstract

2023 NeurIPS SoLaR Workshop Accepted

Towards Auditing Large Language Models: Improving Text-based Stereotype Detection

Key Takeaways

Abstract

Paper Info

Responsible AI Topics

Links

Towards Auditing Large Language Models: Improving Text-based Stereotype Detection

Key Takeaways

Abstract

Paper Info

Responsible AI Topics

Links