VadCLIP++: Dynamic vision-language model for weakly supervised video anomaly detection

Abstract

In the realm of weakly supervised video anomaly detection (WSVAD), the integration of Contrastive Language-Image Pre-training (CLIP) models has demonstrated substantial benefits, highlighting that learning through textual prompts can effectively distinguish between anomalous events and enhance the expression of visual features. However, existing CLIP models in video anomaly detection tasks rely solely on static textual prompts, neglecting the temporal continuity of anomalous behaviors, which limits the understanding of dynamic anomalous behaviors. To address this, this paper proposes a dynamic learnable text prompting mechanism, which supervises and learns the frame difference features of adjacent consecutive frames in the video to capture the dynamic changes of anomalous behaviors. At the same time, by incorporating static textual prompts, the model precisely focuses on anomalous behaviors within individual frames, making the textual prompts in different states more sensitive to the temporal and spatial action details of the video. In addition, a Spatial Feature Selection Module (SFSM) is proposed, which leverages random sampling and TOP-k selection mechanisms to enhance the model's generalization ability in anomalous regions of video frames, while modeling the spatial relationships of the global context. The dynamic learnable text prompt branch handles temporal anomalies by extracting inter-frame difference features, while the static text prompt branch optimizes in-frame anomaly localization under the influence of the SFSM module. The dual-branch collaboration establishes complementary spatiotemporal representations, collectively enhancing detection performance. Experimental results show that on the XD-Violence and UCF-Crime datasets, the proposed method achieves 85.03% AP and 88.12% AUC, thoroughly validating its effectiveness in anomaly detection tasks.