Backdoor Learning in Language Models and Vision-Language Models

Backdoor attacks represent a serious but underexamined security threat in AI systems. The paper identifies backdoor attacks—where models are secretly compromised during training to behave maliciously on specific inputs—as a critical vulnerability in both language models and vision-language models that process both text and images.

Current AI systems lack adequate defenses against these hidden attacks. The research emphasizes that as NLP and vision-language models become more capable, their security vulnerabilities have grown proportionally, suggesting that trustworthiness has not kept pace with capability improvements.

Detection and analysis of backdoors is essential for responsible AI deployment. The thesis focuses on developing methods to identify, analyze, and understand backdoor attacks, which is foundational work needed before these powerful AI systems can be safely deployed in high-stakes applications.

Multimodal AI systems (those combining text and images) face compounded security challenges. Vision-language models present unique vulnerabilities because attackers can exploit multiple input channels—they could hide malicious instructions in images, text, or both—making these systems particularly difficult to secure.

Security and efficiency must be balanced in trustworthy AI development. The paper addresses both backdoor security and efficient representation learning, recognizing that responsible AI requires attention to multiple dimensions beyond just accuracy, including how models are protected and how resources are used.