BiasFairnessAccountabilitystat.ML
Detecting algorithmic bias in medical-AI models using trees
Jeffrey Smith, Andre Holder, Rishikesan Kamaleswaran, Yao Xie
December 5, 2023
arXiv.org
4 citations
Key Takeaways
-
AI bias in healthcare is a real problem that needs practical solutions: The paper addresses a critical gap in medical AI systems—while these tools are increasingly used to support clinical decisions (like sepsis prediction), there's often no systematic way to check if they're treating all patients fairly.
-
A new detection method uses decision trees to find bias: The researchers developed a framework using Classification and Regression Trees (CART) that can identify specific areas where medical-AI models may be making biased predictions, rather than just flagging that bias exists somewhere in the system.
-
The approach was tested in real hospital settings: Beyond controlled experiments, the team validated their bias-detection method using actual patient data from Grady Memorial Hospital in Atlanta, demonstrating that this isn't just theoretical—it can work in actual clinical environments where these AI systems operate.
-
This tool could help ensure equitable patient care: By identifying where algorithmic bias occurs in medical decision support systems, hospitals and healthcare providers can use this framework to audit their AI tools and work toward fairer, more equitable outcomes across different patient populations.
Abstract
With the growing prevalence of machine learning and artificial intelligence-based medical decision support systems, it is equally important to ensure that these systems provide patient outcomes in a fair and equitable fashion. This paper presents an innovative framework for detecting areas of algorithmic bias in medical-AI decision support systems. Our approach efficiently identifies potential biases in medical-AI models, specifically in the context of sepsis prediction, by employing the Classification and Regression Trees (CART) algorithm with conformity scores. We verify our methodology by conducting a series of synthetic data experiments, showcasing its ability to estimate areas of bias in controlled settings precisely. The effectiveness of the concept is further validated by experiments using electronic medical records from Grady Memorial Hospital in Atlanta, Georgia. These tests demonstrate the practical implementation of our strategy in a clinical environment, where it can function as a vital instrument for guaranteeing fairness and equity in AI-based medical decisions.