CF2M-Net: Cross-feature fusion and memory-constraint network for video anomaly detection

Abstract

Video anomaly detection (VAD) aims to automatically identify anomalous events in surveillance videos that are significantly different from the normal pattern. Most existing methods learn the spatial-temporal distribution of normal features and detect deviations as anomalies. Typically, they employ autoencoders to independently learn appearance and motion features, but this separate learning limits the exploitation of their interrelation in real-world scenarios. To enhance the representation of normal patterns by capturing feature interrelation, we propose a cross-feature fusion and memory-constraint network (CF²M-Net) for VAD. Specifically, inspired by the representational ability of cross-attention in multimodal fusion, we design a cross-attention and memory-constraint (CM) module to enrich appearance features with motion information. To prevent overfitting to anomalous events, the memory-constraint module further constrains fused features within the distribution of normal patterns. We design an attention fusion (AF) decoder to predict normal features closer to the normal distribution, enhancing their separability from anomalies. By jointly modeling appearance and motion through feature fusion and memory constraints, CF²M-Net provides more discriminative normal representations for anomaly detection. Experimental evaluations on three benchmark datasets show that the CF²M-Net performs comparably with leading approaches. Moreover, the detailed evaluations indicate the effectiveness of normal representation based appearance-motion fusion features for VAD.