Improving adversarial transferability via semantic-style joint expectation perturbations

Abstract

Style and content information, which are model-independent inherent properties of an image, serve as crucial information that deep neural networks depend on for classification tasks. However, most existing gradient-based attacks mainly distort content-related information through semantic distortion of the model’s final output, neglecting the role of style information. To fully distort the inherent intrinsic information of the image, this paper proposes Semantic-Style joint Expectation Perturbations (SSEPs). Specifically, we first establish a style loss based on the kernel function from the feature space of the surrogate model and inject it into gradient-based attacks to form a Semantics-Style joint Loss (SSL) for generating joint perturbations. Subsequently, we use gradient normalization and the proposed dynamic gradient decomposition scheme to address the problems of multi-objective gradient magnitude differences and gradient conflicts that occur in SSL during optimization. Finally, we generate SSEPs by motivating the maximization of the expected loss, thereby enhancing the transferability of Adversarial Examples (AEs). On the ImageNet sub-dataset, extensive experiments show that AEs covered with SSEPs have high transferability. Compared to the baseline attack (MI-FGSM), our method achieves at least a 14 % and 5 % higher attack success rate for normally trained models and defense models, respectively. Compared with other classic and advanced gradient-based attacks and feature-level attacks, our method still has advantages in attack performance. Our code is available at: https://github.com/OUTOFTEN/TransferAttack-ssep