FTA2C: Achieving superior trade-off between accuracy and robustness in adversarial training

Abstract

Deep neural networks are notoriously vulnerable to adversarial perturbations, largely due to the presence of non-robust features that destabilize model performance. Traditional Adversarial Training (AT) methods on feature space typically operate on one part of features individually, resulting in the loss of useful information in them, and improve robustness at the expense of accuracy, making it difficult to optimize the inherent trade-off between the two. To address this challenge, we propose a novel plug-in method termed Feature Transformation Alignment and Compression (FTA2C). FTA2C comprises three key components. First, a compression network constrains the perturbation space to reduce the vulnerability of non-robust features. Second, a feature transformation network enhances the expressiveness of robust features. Third, an alignment mechanism enforces consistency between adversarial and natural samples in the robust feature space. The above mechanism achieves co-processing of the two parts of the feature. Additionally, we propose the Defense Efficiency Metric (DEM) to evaluate defense methods. DEM quantifies the trade-off between maintaining natural accuracy and enhancing adversarial robustness, offering a unified and interpretable standard for comparing defense strategies. Extensive experiments conducted on four benchmark datasets demonstrate that FTA2C significantly improvements robustness under the high-level accuracy, resulting in superior trade-off performance. Our code is available at https://github.com/HymanGao31/FTA2C.