Effect of Data Augmentation on Conformal Prediction for Diabetic Retinopathy

Data augmentation choices directly impact the reliability of uncertainty estimates in medical AI: The study found that different augmentation strategies (geometric transforms, CLAHE, Mixup, CutMix) produced measurably different results in conformal prediction metrics, meaning the same model architecture could generate more or less trustworthy confidence measures depending on how training data was modified.

Sample-mixing techniques like Mixup and CutMix improve both accuracy and uncertainty quantification: These augmentation methods not only helped models make better predictions but also produced more reliable and efficient uncertainty estimates—prediction sets that were both accurate and appropriately sized—suggesting they should be prioritized for high-stakes medical applications.

Standard augmentation practices can undermine trustworthiness guarantees: CLAHE (a common image enhancement technique) negatively impacted model certainty, demonstrating that widely-used training practices may inadvertently compromise the statistical guarantees that make AI systems trustworthy for clinical deployment.

Uncertainty quantification must be considered during model development, not as an afterthought: The findings emphasize that augmentation strategies should be deliberately chosen alongside uncertainty quantification methods rather than selected independently, requiring closer collaboration between model developers and those responsible for validating clinical reliability.

Conformal prediction offers verifiable safety for diabetic retinopathy screening: By generating prediction sets with statistical coverage guarantees rather than single-point predictions, conformal prediction provides a practical framework for deploying deep learning in high-stakes medical settings where knowing the limits of model confidence is critical.