A Lightweight Defense Against Code Poisoning Attacks Based on Code Naturalness

Abstract

Neural code models (NCMs) have demonstrated extraordinary capabilities in code intelligence tasks. Meanwhile, the security of NCMs and NCMs-based systems has garnered increasing attention. In particular, NCMs are often trained on large-scale data from potentially untrustworthy sources, providing attackers with the opportunity to manipulate them by inserting crafted samples into the data. This type of attack is called a code poisoning attack (also known as a backdoor attack). It allows attackers to implant backdoors in NCMs and thus control model behavior, which poses a significant security threat. However, there is still a lack of effective techniques for detecting various complex code poisoning attacks. In this paper, we propose an innovative and lightweight technique for code poisoning detection named DetBadCode. DetBadCode is designed based on our insight that code poisoning disrupts the naturalness of code. Specifically, DetBadCode first builds a code language model (CodeLM) on a lightweight ${\boldsymbol n}$-gram language model. Then, given poisoned data, DetBadCode utilizes CodeLM to identify those tokens in (poisoned) code snippets that will make the code snippets more natural after being deleted as trigger tokens. Considering that the removal of some normal tokens in a single sample might also enhance code naturalness, leading to a high false positive rate (FPR), we aggregate the cumulative improvement of each token across all samples. Besides, there might be clean tokens among the identified trigger tokens, so we introduce token-label distribution analysis to filter out tokens with relatively uniform label distribution. Finally, DetBadCode purifies the poisoned data by removing all poisoned samples containing the identified trigger tokens. Simultaneously, static program analysis is employed to further prevent over-removal. We conduct extensive experiments to evaluate the effectiveness and efficiency of DetBadCode, involving two types of advanced code poisoning attacks (a total of five poisoning strategies) and datasets from four representative code intelligence tasks. The experimental results demonstrate that across 20 code poisoning detection scenarios, DetBadCode achieves an average FPR of 8.30% and an average Recall of 100%, significantly outperforming four baselines. More importantly, DetBadCode is very efficient, with a minimum time consumption of only 5 minutes, and is 25 times faster than the best baseline on average.