Study of Distractors in Neural Models of Code

Neural code models can be manipulated by seemingly irrelevant features: The study found that removing certain tokens significantly impacts model confidence in predictions, suggesting these "distractor" features can undermine a model's certainty even when they shouldn't logically matter—raising concerns about model robustness in code analysis tasks.

Understanding what confuses AI models is as important as understanding what helps them: While most explainability research focuses on features that support predictions, this work demonstrates that identifying "distractor" features provides complementary insights into how neural models actually make decisions, offering a more complete picture of model behavior.

Token categories matter significantly for model confidence: The research reveals that different types of tokens (such as specific syntax elements or variable names) play varying roles in affecting model confidence, suggesting that models may rely on superficial patterns rather than deep code understanding.

Current neural code models lack transparency in their decision-making: By identifying features that distract or confuse models, this work highlights gaps in how well we understand why these systems make their predictions, which is critical for deploying them responsibly in software development and security applications.

Distractor analysis could improve AI safety in code-related tasks: Understanding which features inappropriately influence model confidence could help developers build more robust code analysis systems and identify when models are making predictions for the wrong reasons—important for high-stakes applications like vulnerability detection.