A review of backdoor attacks and defenses in code large language models: Implications for security measures

Abstract

Context:

Large Language Models (LLMS) have revolutionized software engineering by bridging human language understanding and complex problem solving. However, resource constraints often lead users to rely on open-source models or third-party platforms for training and prompt engineering, introducing significant security vulnerabilities.

Objective:

This study provides a comprehensive analysis of backdoor attacks targeting LLMS in software engineering, with a particular focus on fine-tuning methods. Our work addresses a critical gap in existing literature by proposing a novel three-category framework for backdoor attacks: full-parameter fine-tuning, parameter-efficient fine-tuning, and no-tuning attacks.

Methods:

We systematically reviewed existing studies and analyzed attack success rates across different methods. Full-parameter fine-tuning generally achieves high success rates but requires significant computational resources. Parameter-efficient fine-tuning offers comparable success rates with lower resource demands, while no-tuning attacks exhibit variable success rates depending on prompt design, posing unique challenges due to their minimal resource requirements.

Results:

Our findings underscore the evolving landscape of backdoor attacks, highlighting the shift towards more resource-efficient and stealthy methods. These trends emphasize the need for advanced detection mechanisms and robust defense strategies.

Conclusion:

By focusing on code-specific threats, this study provides unique insights into securing LLMS in software engineering. Our work lays the foundation for future research on developing sophisticated defense mechanisms and understanding stealthy backdoor attacks.