SecuritySafetyAccountabilitycs.LG
Machine Learning with Electronic Health Records is vulnerable to Backdoor Trigger Attacks
Byunggill Joe, Akshay Mehra, Insik Shin, Jihun Hamm
June 15, 2021
arXiv.org
12 citations
Key Takeaways
-
Healthcare AI systems can be secretly manipulated through poisoned training data: Attackers can compromise machine learning models used in hospitals by tainting less than 5% of electronic health records during the training phase, allowing them to selectively alter predictions about patient mortality with 97% success rates across different model types.
-
Backdoor attacks are difficult to detect because poisoned data looks normal: The researchers created malicious data that has "statistically similar features to the original data," meaning standard data quality checks may fail to identify compromised records, creating a hidden vulnerability in healthcare AI systems.
-
Multiple hospital systems could be vulnerable simultaneously: The attack works against different types of machine learning models (Logistic Regression, Multilayer Perceptron, and Long Short-term Memory) without requiring attackers to know which specific model a hospital uses, suggesting widespread risk across healthcare institutions.
-
Critical clinical decisions could be compromised: Since these models are used for mortality prediction and hospital resource management, successful backdoor attacks could lead to incorrect patient risk assessments and misallocation of medical resources, with potentially serious consequences for patient care.
-
Current safeguards may be insufficient: The paper demonstrates a significant gap between how robust healthcare AI systems are assumed to be and their actual vulnerability to sophisticated attacks, suggesting that hospitals need stronger security measures beyond standard model validation.