FairnessAccountabilityTransparencycs.HC
Justified Evidence Collection for Argument-based AI Fairness Assurance
Alpay Sabuncuoglu, Christopher Burr, Carsten Maple
May 12, 2025
Conference on Fairness, Accountability and Transparency
7 citations
Key Takeaways
-
Fairness assurance requires structured, evidence-based arguments: The paper demonstrates that ensuring AI fairness isn't just about good intentions—it requires systematic documentation of claims about fairness paired with concrete evidence, similar to how safety cases work in engineering disciplines.
-
Multi-stakeholder involvement from the start is essential: The framework emphasizes that during the requirements planning phase, diverse teams (including domain experts, ethicists, and stakeholders) must collectively define fairness goals before development begins, rather than treating fairness as an afterthought.
-
Fairness monitoring must be continuous, not one-time: Rather than checking fairness only at deployment, the paper's approach uses dynamic monitoring that continuously gathers evidence from system artifacts (like automated test metrics and model documentation) throughout the AI system's entire lifecycle, from development through deprovisioning.
-
Evidence should come from existing system documentation: The framework reduces burden by automatically collecting evidence from materials already being created—such as data documentation, model cards, and test results—rather than requiring entirely new compliance processes.
-
Fairness governance is a sociotechnical challenge: The paper emphasizes that ensuring fair AI systems requires careful deliberation across all development stages and involves both technical considerations (metrics, data quality) and broader organizational and stakeholder concerns, not just algorithmic fixes.