A Survey of Trojans in Neural Models of Source Code: Taxonomy and Techniques

Neural code models can be deliberately poisoned to behave maliciously: The paper establishes that AI models trained on source code can be manipulated through "Trojan" attacks—injecting hidden triggers that cause the model to produce incorrect or harmful outputs when activated, similar to backdoors in software.

Trojans in code models pose real security risks: Since these neural models are increasingly used for code generation, completion, and vulnerability detection, poisoned models could introduce bugs, security flaws, or malicious code into software development pipelines without developers realizing the AI system has been compromised.

Triggers can be subtle and hard to detect: The paper's aspect-based classification of triggers reveals that attackers can embed hidden activation patterns in code that are difficult for humans to spot, making it challenging to identify when a model has been poisoned through standard code review processes.

Current safeguards are insufficient: By surveying existing poisoning strategies and techniques, the paper highlights gaps in how we currently protect neural code models, suggesting that developers and organizations using AI for code need stronger verification and monitoring practices.

Responsible AI requires understanding model vulnerabilities: The paper contributes to Safe AI research by mapping out how code models can fail, which is essential for building more trustworthy AI systems and establishing better practices for validating that code-generating AI systems haven't been compromised before deployment.