A Simple Yet Practical Backdoor Prompt Attack Against Black-Box Code Summarization Engines

A code summarization engine based on large language models (LLMs) can describe code functionality from different perspectives according to programmers' needs. However, these engines are at risk of black-box backdoor attacks. We propose a simple yet practical method called Bad Prompt Attack (BPA), specifically designed to investigate such black-box backdoor attacks. This innovative attack method aims to induce the code summarization engine to generate summarizations that conceal security vulnerabilities in source code. Consistent with most commercial code summarization engines, BPA only assumes black-box query access to the target engine without requiring knowledge of its internal structure. This attack targets in-context learning by injecting adversarial demonstrations into user input prompts. We validated our method on the SOTA black-box commercial service, OpenAI API. In security-critical test cases covering seven types of CWE, BPA significantly increased the likelihood that the code summarization engine would generate the attacker-desired code summarization targets, achieving an average attack success rate (ASR) of 91.4%. This result underscores the potential threat of backdoor attacks on code summarization tasks while providing essential reference points for future defense research.