RadEdit: stress-testing biomedical vision models via diffusion image editing

Small, biased datasets are a major problem in biomedical AI: The paper identifies that biomedical imaging datasets are often limited in size and contain biases, causing real-world model performance to drop significantly below what internal testing suggests—a critical gap that could affect patient care.

Generative image editing can test AI models before deployment: Rather than waiting for real-world failures, researchers can use AI-generated synthetic variations to stress-test biomedical vision models and identify failure modes in advance, potentially reducing costs and patient harm before systems are deployed clinically.

Current editing methods can introduce misleading correlations: Existing image editing approaches can accidentally create spurious patterns (like always showing treatment when disease is present), which limits their reliability for testing—highlighting the need for more careful technical approaches to this problem.

RadEdit constrains edits to prevent unrealistic changes: The proposed method uses multiple masks to ensure edited images remain realistic and consistent, addressing the spurious correlation problem and making the stress-testing more trustworthy for real-world applicability.

Three types of dataset shifts can be systematically tested: The paper demonstrates that models can be evaluated against acquisition shifts (different imaging equipment), manifestation shifts (how disease appears), and population shifts (different patient demographics)—providing a structured framework for assessing robustness without collecting new data.