BiasFairnessAccountabilitycs.CY
Reducing Biases towards Minoritized Populations in Medical Curricular Content via Artificial Intelligence for Fairer Health Outcomes
Chiman Salavati, Shannon Song, Willmar Sosa Diaz, Scott A. Hale, Roberto E. Montenegro, Fabricio Murai, Shiri Dori-Hacohen
May 21, 2024
AAAI/ACM Conference on AI, Ethics, and Society
9 citations
Key Takeaways
-
Medical textbooks contain persistent biases that harm minoritized populations: The paper documents that biased information—including stereotypes related to gender, race, ethnicity, age, and geography—continues to be taught in medical schools long after being scientifically debunked, potentially contributing to healthcare disparities.
-
AI can help identify bias at scale, but requires human oversight: BRICC uses machine learning to flag potentially biased content in medical materials, achieving 92.3% accuracy on general bias detection. Critically, the system operates in an "expert-in-the-loop" fashion, meaning human medical experts review AI findings rather than relying on automation alone—a responsible approach to high-stakes decisions.
-
The dataset and annotation process are foundational to responsible AI: The researchers built a gold-standard dataset of over 12,000 pages meticulously reviewed by medical experts using comprehensive coding guidelines. This careful human annotation work is essential for training trustworthy AI systems and demonstrates that quality data matters more than model complexity.
-
Different types of bias require different detection approaches: The paper found that specialized classifiers trained on specific bias types (gender, race, etc.) outperformed a single general model, suggesting that one-size-fits-all AI solutions may miss important nuances in bias detection across different populations.
-
This work addresses a real equity problem in medical education: By systematizing bias detection, BRICC could help accelerate the removal of harmful stereotypes from curricula, potentially improving health outcomes for minoritized communities who have historically received lower-quality medical care.
Abstract
Biased information (recently termed bisinformation) continues to be taught in medical curricula, often long after having been debunked. In this paper, we introduce BRICC, a firstin-class initiative that seeks to mitigate medical bisinformation using machine learning to systematically identify and flag text with potential biases, for subsequent review in an expert-in-the-loop fashion, thus greatly accelerating an otherwise labor-intensive process. A gold-standard BRICC dataset was developed throughout several years, and contains over 12K pages of instructional materials. Medical experts meticulously annotated these documents for bias according to comprehensive coding guidelines, emphasizing gender, sex, age, geography, ethnicity, and race. Using this labeled dataset, we trained, validated, and tested medical bias classifiers. We test three classifier approaches: a binary type-specific classifier, a general bias classifier; an ensemble combining bias type-specific classifiers independently-trained; and a multitask learning (MTL) model tasked with predicting both general and type-specific biases. While MTL led to some improvement on race bias detection in terms of F1-score, it did not outperform binary classifiers trained specifically on each task. On general bias detection, the binary classifier achieves up to 0.923 of AUC, a 27.8% improvement over the baseline. This work lays the foundations for debiasing medical curricula by exploring a novel dataset and evaluating different training model strategies. Hence, it offers new pathways for more nuanced and effective mitigation of bisinformation.
Accepted at the 2024 AAAI/ACM Conference on AI, Ethics and Society (AIES'24)
Paper Info
arXiv ID2407.12680
PublishedMay 21, 2024
RevisedAugust 27, 2025
Citations
9
Relevance88%
VenueAAAI/ACM Conference on AI, Ethics, and Society
TypeJournalArticle, Review
Categorycs.CY
Responsible AI Topics
BiasFairnessAccountability
Links
Citation data from Semantic Scholar