Learning Interpretable and Transferable Representations via Wavelet-Constrained Transformer for Industrial Acoustic Diagnosis

Abstract

With the rapid development of sensing and computing technology, transfer learning has become increasingly favored for mechanical fault diagnosis due to its ability to handle distribution differences across different domains. The interpretability of backbone models used in transfer learning, such as convolutional neural network (CNN), recurrent neural network (RNN), graph neural network (GNN), and transformer, is, however, limited, hindering their acceptance and adoption by industrial users. In order to address this problem, we propose an interpretable wavelet-constrained transformer for diagnostic tasks designed to extract local features and aggregate global information. Specifically, our model applies the dual-tree complex wavelet constraint to the transformer structure, ensuring approximate shift invariance. This improves diagnostic accuracy while reducing the number of parameters. Additionally, we explore the Einstein summation (ES) for matrix multiplication in frequency band blending after wavelet transforms to reduce computational complexity and accelerate convergence speed. In order to enhance the model’s transferability across different domains, we incorporate uncertainty-constrained loss on the model output using temperature scaling and uncertainty reweighting. This effectively reduces class confusion and improves accuracy in the target domain. Considering the necessity of noncontact measurement in mechanical systems for real-world applications, we use acoustics signals to verify the effectiveness of our transferable and interpretable model. The experimental results show that, compared with other commonly used models, our model significantly improves cross-domain diagnostic accuracy without affecting interpretability.