Write It Like You See It: Detectable Differences in Clinical Notes By Race Lead To Differential Model Recommendations

Models can detect race from clinical notes even when explicit racial identifiers are removed. The researchers found that machine learning algorithms could identify patient self-reported race from clinical notes that had been stripped of direct racial references, suggesting that subtle linguistic patterns or clinical descriptions contain detectable racial information.

Humans cannot reliably detect what machines can. While ML models successfully identified patient race from redacted notes, human experts were unable to accurately predict race from the same documents, highlighting a critical gap between human and algorithmic perception of bias in medical data.

Removing explicit racial information is insufficient to prevent bias. Even when clinical notes were deliberately redacted of direct racial identifiers, models trained on these notes still perpetuated existing biases in clinical treatment recommendations, demonstrating that implicit racial information embedded in language patterns can drive discriminatory outcomes.

Clinical language itself may encode racial disparities. The findings suggest that differences in how clinical notes are written about patients of different races—potentially reflecting real disparities in clinical care or communication—become detectable patterns that algorithms can exploit, even unintentionally.

Current debiasing approaches may be inadequate. The study reveals that standard practices like removing explicit demographic information from training data are insufficient safeguards, suggesting healthcare organizations need more sophisticated strategies to identify and mitigate bias in ML systems used for clinical decision-making.