Predictive Representativity: Uncovering Racial Bias in AI-based Skin Cancer Detection

Racial bias persists in skin cancer AI despite balanced datasets: The study found that AI classifiers trained on the HAM10000 dataset—which had proportional representation of different skin types—still significantly underperformed for individuals with darker skin, demonstrating that simply including diverse data doesn't guarantee fair outcomes.

"Representativity" requires measuring real-world performance, not just data composition: The paper introduces Predictive Representativity as a framework that shifts focus from whether datasets look diverse to whether AI systems actually perform equitably across different populations in practice, offering a more meaningful measure of fairness.

Independent testing revealed hidden disparities: When the classifiers were tested on an independent clinical dataset from Colombia (BOSQUE Test set), performance gaps by skin phototype became apparent, highlighting the importance of validating AI systems on diverse, real-world data beyond their training sources.

Post-hoc auditing and transparency are essential safeguards: The authors emphasize that fairness auditing after AI systems are developed—combined with transparent documentation of datasets and inclusive validation processes—is critical for identifying and addressing structural inequities before deployment in clinical settings.

Current practices may mask inequitable outcomes: The findings suggest that existing approaches to AI validation in medical contexts may fail to catch bias against marginalized populations, underscoring the need for more rigorous, equity-focused evaluation standards across the healthcare AI industry.