Explainable AI and susceptibility to adversarial attacks: a case study in classification of breast ultrasound images

Explainability tools can be manipulated without changing predictions: The study demonstrates that adversarial attacks can dramatically alter the visual explanations (importance maps) that AI systems generate to justify their decisions, even when the final classification remains unchanged. This means doctors could be misled about why an AI made a diagnosis, even if the diagnosis itself appears correct.

Current explanation methods have security vulnerabilities: Techniques like GRAD-CAM, which are increasingly used to make AI decisions interpretable in medical imaging, can be fooled by imperceptible alterations to images. These attacks are "practically undetectable" to human observers, raising concerns about the reliability of explanations in clinical settings where they guide treatment decisions like biopsy location.

Explainability alone doesn't guarantee trustworthiness: The paper highlights a critical gap in responsible AI: making a system's reasoning visible doesn't automatically make it safe or reliable. Even when doctors can "see" what the AI is focusing on, that explanation could be fundamentally misleading due to adversarial manipulation.

Medical AI systems need additional safeguards beyond accuracy: While the researchers achieved competitive classification accuracy using Multi-Task Learning approaches, the work emphasizes that accuracy metrics alone are insufficient for clinical deployment. Systems must be tested for robustness against adversarial attacks and validated to ensure their explanations are trustworthy.

Caution needed when using AI explanations for clinical decisions: The findings suggest that importance maps should not be relied upon as the sole basis for clinical decisions (such as determining biopsy sites) without additional validation, as these explanations can be manipulated independently of the classification outcome.