Diversified perturbation guided by optimal target code for cross-modal adversarial attack

Cross-modal retrieval models are vulnerable to adversarial samples, thus exploring efficient attack methods can help researchers understand the essence of adversarial attack, evaluate the robustness of models, and promote the development of more reliable models. Although existing adversarial attack methods have achieved promising results, how to further improve the transferability of adversarial examples remains an open question. In this paper, we propose a novel transferable targeted attack method. First, we introduce an optimal target code optimization strategy to obtain representative target codes. Subsequently, when generating adversarial examples, we propose a random perturbation strategy to diversify the potential input patterns of perturbations by introducing randomness, thus automatically enhancing the generalization of samples. overcoming the limitations of existing methods that depend on specific image augmentation techniques. Experiments show that this framework can generate highly transferable adversarial samples, for example, when transferring attacks from VGG-F to ResNet50, the proposed method outperforms the SOTA by 10.41% in the I2T task on the MS-COCO dataset; in addition, samples generated by small models can also successfully attack large models, which will help researchers study the existence of adversarial attacks from a new perspective.