High-frequency-based multi-spectral attention for domain generalization

Abstract

Deep learning models have made great progress in many vision tasks, but they suffer from domain shift problem when exposed to out-of-distribution scenarios. Domain generalization (DG) is proposed to learn a model from several observable source domains that can generalize well to unknown target domains. Although recent advances in DG works have achieved promising performance, there is a high demand for computational resource, especially those that employ meta-learning or ensemble learning strategies. However, some pioneering works propose to replace convolutional neural network (CNN) as the backbone architecture with multi-layer perceptron (MLP)-like models that can not only learn long-range spatial dependencies but also reduce network parameters using Fourier transform-based techniques. Inspired by this, in this paper, we propose a high-frequency-based multi-spectral attention (HMCA) to facilitate a lightweight MLP-like model to learn global domain-invariant features by focusing on high-frequency components sufficiently. Moreover, we adopt a data augmentation strategy based on Fourier transform to simulate domain shift, thus enabling the model to pay more attention on robust features. Extensive experiments on benchmark datasets demonstrate that our method is superior to the existing CNN-based and MLP-based DG methods.