CodeSearchAttack: Enhancing soft-label black-box adversarial attacks on code

Adversarial attacks on code data face significant challenges due to its discrete and non-differentiable nature. Soft-label black-box code adversarial attacks, in particular, are a highly complex task, with research in this area still in its early stages. Existing methods leave room for improvement in performance. For instance, greedy search-based attacks often get trapped in local optima, resulting in excessive perturbations. To tackle these challenges, we propose a novel framework, CodeSearchAttack, for crafting high-quality adversarial examples. CodeSearchAttack leverages constrained K-means to identify diverse substitutions in the variable embedding space and employs an improved beam search to craft adversarial examples. Additionally, it calculates variable importance using information derived from soft labels. Experiments on four code classification tasks demonstrate that CodeSearchAttack significantly outperforms state-of-the-art baseline methods. Under a query budget of 100, CodeSearchAttack achieves superior attack efficacy compared to existing soft-label attacks.