A dual-reconstruction self-rectification framework with momentum memory-augmented network for multivariate time series anomaly detection

Abstract

The discrepancy between the actual contaminated data and the normality assumption poses a serious challenge to existing methods that rely on clean training data. For methods that consider contamination, the mainstream model-level methods with memory-augmented structures struggle with biased similarity measures and fail to utilize historical information, leading to inaccurate reconstruction of latent variables. Most training-level methods may confuse contaminated data with hard-to-learn normal data, affecting the model’s ability to learn normal patterns. Moreover, there is a lack of using adjustment loss to effectively constrain model-level methods to learn normal data while suppressing contaminated data, which limits the further improvement of anomaly detection performance. This paper proposes a Dual-Reconstruction Self-Rectification framework with Momentum Memory-augmented network based on Transformer (DRMoMe) for multivariate time series anomaly detection. At the model level, a momentum memory module based on Transformer is proposed, which employs the momentum-updated framework to align the representation space and designs the multihead-attention mechanism with the similarity-based update strategy to ensure the accuracy and diversity of the memory vectors. At the training level, this paper designs a self-rectification framework, which uses the difference between dual-reconstruction paths as the loss adjustment weights to adjust the model’s learning dynamically. Additionally, the method uses the characteristic of the memory module to amplify the weight difference between the contaminated and normal data, effectively integrating the model-level and training-level approach to help the model focus on learning the normal pattern. The DRMoMe outperforms 21 state-of-the-art baselines in experiments conducted on five benchmark datasets from different domains.