VIWHard: Text adversarial attacks based on important-word discriminator in the hard-label black-box setting

Abstract

In the hard-label black-box setting, the adversary only obtains the decision of the target model, which is more practical. Both the perturbed words and the sets of substitute words affect the performance of adversarial attack. We propose a hard-label black-box adversarial attack framework called VIWHard, which takes important words as perturbed words. In order to verify the words which highly impact on the classification of the target model, we design an important-word discriminator consisting of a binary classifier and a masked language model as an important component of VIWHard. Meanwhile, we use a masked language model to construct the context-preserving sets of substitute words for important words, which further improves the naturalness of the adversarial texts. We conduct experiments by attacking WordCNN, WordLSTM and BERT on seven datasets, which contain text classification, toxic information, and sensitive information datasets. Experimental results show that our method achieves powerful attacking performance and generates natural adversarial texts. The average attack success rate on the seven datasets reaches 98.556%, and the average naturalness of the adversarial texts reaches 7.894. Specially, on the four security datasets Jigsaw2018, HSOL, EDENCE, and FAS, our average attack success rate reaches 97.663%, and the average naturalness of the adversarial texts reaches 8.626. In addition, we evaluate the attack performance of VIWHard on large language models (LLMs), the generated adversarial examples are effective for LLMs.