Evaluating the Impact of Medical Image Reconstruction on Downstream AI Fairness and Performance

Reconstruction quality metrics don't predict diagnostic accuracy: The study found that conventional pixel-level metrics like PSNR (Peak Signal-to-Noise Ratio) poorly track whether downstream diagnostic AI models perform well, meaning images that look worse by traditional measures may still enable accurate diagnoses.

Reconstruction can amplify demographic biases, particularly by sex: Fairness metrics showed greater variability than performance metrics, with reconstruction models sometimes amplifying existing biases—particularly regarding patient sex—though the additional bias introduced was modest compared to biases already present in diagnostic models.

Diagnostic accuracy remains stable across noise levels: Despite declining image quality, diagnostic performance for both classification and segmentation tasks remained largely stable even as reconstruction PSNR decreased with increasing image noise, suggesting current diagnostic AI is relatively robust to reconstruction quality variations.

Current bias mitigation strategies have limited effectiveness: When the researchers adapted two bias mitigation strategies from classification literature to the reconstruction setting, they observed limited success, suggesting that existing debiasing approaches may not directly transfer to the image reconstruction context.

Holistic evaluation throughout the imaging workflow is essential: The findings underscore that AI systems in medical imaging should be evaluated not just on isolated components (like reconstruction quality) but on their combined impact on both diagnostic performance and fairness across demographic groups.