FairnessBiasEthicscs.CY
AI Fairness in Practice
David Leslie, Cami Rincon, Morgan Briggs, Antonella Perini, Smera Jayadeva, Ann Borda, SJ Bennett, Christopher Burr, Mhairi Aitken, Michael Katell, Claudia Fischer, Janis Wong, Ismael Kherroubi Garcia
February 19, 2024
Social Science Research Network
14 citations
1 influential
Key Takeaways
-
Fairness in AI lacks a universal definition, but there is broad agreement that the core principle should center on equality and non-discrimination—specifically protecting people from both direct and indirect discrimination and discriminatory harassment.
-
Context matters significantly: Different stages of an AI project lifecycle create different fairness challenges, requiring teams to apply a "multi-lens approach" that considers Data Fairness, Application Fairness, Model Design and Development Fairness, Metric-Based Fairness, System Implementation Fairness, and Ecosystem Fairness separately.
-
A society-centered approach is essential: Rather than imposing a single definition of fairness, the paper advocates for understanding fairness through the lens of specific contexts and societal values, helping project teams better identify and mitigate bias and discrimination throughout the entire AI workflow.
-
Fairness requires both ethical and legal grounding: The paper establishes that fairness concerns should be evaluated against both ethical principles and legal criteria for non-discrimination, providing a framework for teams to systematically identify where unfair bias might emerge.
Abstract
Reaching consensus on a commonly accepted definition of AI Fairness has long been a central challenge in AI ethics and governance. There is a broad spectrum of views across society on what the concept of fairness means and how it should best be put to practice. In this workbook, we tackle this challenge by exploring how a context-based and society-centred approach to understanding AI Fairness can help project teams better identify, mitigate, and manage the many ways that unfair bias and discrimination can crop up across the AI project workflow. We begin by exploring how, despite the plurality of understandings about the meaning of fairness, priorities of equality and non-discrimination have come to constitute the broadly accepted core of its application as a practical principle. We focus on how these priorities manifest in the form of equal protection from direct and indirect discrimination and from discriminatory harassment. These elements form ethical and legal criteria based upon which instances of unfair bias and discrimination can be identified and mitigated across the AI project workflow. We then take a deeper dive into how the different contexts of the AI project lifecycle give rise to different fairness concerns. This allows us to identify several types of AI Fairness (Data Fairness, Application Fairness, Model Design and Development Fairness, Metric-Based Fairness, System Implementation Fairness, and Ecosystem Fairness) that form the basis of a multi-lens approach to bias identification, mitigation, and management. Building on this, we discuss how to put the principle of AI Fairness into practice across the AI project workflow through Bias Self-Assessment and Bias Risk Management as well as through the documentation of metric-based fairness criteria in a Fairness Position Statement.