Adversarial robustness of a U-Net-based model observer for CT protocol optimization

AI medical imaging systems can be fooled by imperceptible changes: The study found that tiny, invisible perturbations to CT images caused a U-Net model to misclassify findings up to 75% of the time, raising concerns about whether AI systems are truly reliable for clinical decision-making.

Different AI tasks have different vulnerability levels: While the model's ability to detect abnormalities was highly susceptible to adversarial attacks (75% failure rate), its ability to locate where abnormalities were found proved more robust (42% failure rate), suggesting that some medical AI functions may be inherently more trustworthy than others.

Adversarial training can substantially improve robustness: When researchers trained the model using adversarial examples, they reduced attack success rates from 50% down to just 7% for detection tasks, demonstrating that vulnerabilities can be meaningfully addressed without sacrificing the model's normal performance.

Current testing methods may be insufficient for clinical deployment: The study reveals that standard performance metrics alone don't capture whether medical AI systems will fail under real-world conditions, suggesting that adversarial robustness testing should become a standard part of validating AI systems before clinical use.

Texture analysis reveals how attacks work: The research found that successful adversarial attacks consistently altered specific texture features in images while leaving overall image statistics unchanged, providing insights into how to better design and monitor AI systems for medical imaging.