AI2030 Responsible AI Tools Marketplace

Key Takeaways

Fairness cannot be determined by technical measures alone. The paper demonstrates that there is no universally accepted fairness measure and that fairness is context-dependent, meaning that involving stakeholders in defining what "fair" means for their specific situation is essential for responsible AI systems.
Lay people can effectively provide fairness feedback on AI decisions. In Study 1, 58 non-expert users were able to evaluate specific loan application decisions and provide meaningful feedback on fairness, suggesting that stakeholders without AI backgrounds can meaningfully contribute to improving AI systems when given appropriate opportunities.
Stakeholder feedback can improve both fairness and accuracy. The researchers found that retraining AI models based on user feedback about fairness had measurable effects on both fairness metrics and accuracy, indicating that incorporating stakeholder perspectives doesn't necessarily require trading off one for the other.
Showing stakeholders the impact of their feedback increases engagement. Study 2 revealed that when participants saw how their feedback affected fairness outcomes, it deepened their understanding and engagement with the process, suggesting that transparency about the consequences of stakeholder input is important for meaningful participation.
Practical implementation challenges remain significant. While the research demonstrates promise, the paper acknowledges ongoing questions about how best to support stakeholders in providing fairness feedback and how to systematically integrate diverse stakeholder perspectives into AI model development.

Abstract

Fairness is a growing concern for high-risk decision-making using Artificial Intelligence (AI) but ensuring it through purely technical means is challenging: there is no universally accepted fairness measure, fairness is context-dependent, and there might be conflicting perspectives on what is considered fair. Thus, involving stakeholders, often without a background in AI or fairness, is a promising avenue. Research to directly involve stakeholders is in its infancy, and many questions remain on how to support stakeholders to feedback on fairness, and how this feedback can be integrated into AI models. Our work follows an approach where stakeholders can give feedback on specific decision instances and their outcomes with respect to their fairness, and then to retrain an AI model. In order to investigate this approach, we conducted two studies of a complex AI model for credit rating used in loan applications. In study 1, we collected feedback from 58 lay users on loan application decisions, and conducted offline experiments to investigate the effects on accuracy and fairness metrics. In study 2, we deepened this investigation by showing 66 participants the results of their feedback with respect to fairness, and then conducted further offline analyses. Our work contributes two datasets and associated code frameworks to bootstrap further research, highlights the opportunities and challenges of employing lay user feedback for improving AI fairness, and discusses practical implications for developing AI applications that more closely reflect stakeholder views about fairness.

Key Takeaways

Fairness cannot be determined by technical measures alone. The paper demonstrates that there is no universally accepted fairness measure and that fairness is context-dependent, meaning that involving stakeholders in defining what "fair" means for their specific situation is essential for responsible AI systems.
Lay people can effectively provide fairness feedback on AI decisions. In Study 1, 58 non-expert users were able to evaluate specific loan application decisions and provide meaningful feedback on fairness, suggesting that stakeholders without AI backgrounds can meaningfully contribute to improving AI systems when given appropriate opportunities.
Stakeholder feedback can improve both fairness and accuracy. The researchers found that retraining AI models based on user feedback about fairness had measurable effects on both fairness metrics and accuracy, indicating that incorporating stakeholder perspectives doesn't necessarily require trading off one for the other.
Showing stakeholders the impact of their feedback increases engagement. Study 2 revealed that when participants saw how their feedback affected fairness outcomes, it deepened their understanding and engagement with the process, suggesting that transparency about the consequences of stakeholder input is important for meaningful participation.
Practical implementation challenges remain significant. While the research demonstrates promise, the paper acknowledges ongoing questions about how best to support stakeholders in providing fairness feedback and how to systematically integrate diverse stakeholder perspectives into AI model development.

Abstract

Human-in-the-loop Fairness: Integrating Stakeholder Feedback to Incorporate Fairness Perspectives in Responsible AI

Key Takeaways

Abstract

Paper Info

Responsible AI Topics

Links

Human-in-the-loop Fairness: Integrating Stakeholder Feedback to Incorporate Fairness Perspectives in Responsible AI

Key Takeaways

Abstract

Paper Info

Responsible AI Topics

Links