Self-supervised learning video anomaly detection based on time interval prediction and noise classification

Abstract

Video Anomaly Detection (VAD) aims to automatically identify anomalous events in videos that significantly deviate from normal behavioral patterns. Self-supervised learning motivates models to learn effective features from unlabeled data by designing proxy tasks. However, existing approaches often rely on coarse-grained modeling, focusing mainly on global sequence order or holistic scene structures, which may limit their ability to capture subtle motion changes or localized anomalies. Therefore, this paper proposes a self-supervised learning framework combined with fine-grained spatio-temporal proxy tasks to extract key features more accurately. For the temporal branch, we design a time interval prediction task: given a fixed middle frame and randomly sampled frames from both sides, the model predicts their temporal intervals relative to the center frame, thereby modeling the dynamic patterns of behavior. To enhance temporal modeling capabilities, we introduce a multi-head self-attention mechanism to capture inter-frame dependencies in the input sequence. The spatial branch employs a noise classification task inspired by diffusion models, where varying levels of noise are added to image patches, and the model predicts the corresponding noise levels. This encourages learning of local appearance features and patch-level sensitivity to perturbations. Our method is trained in an end-to-end manner and does not rely on pre-trained models. Experiments on three benchmark datasets demonstrate stable performance: the method achieves AUC scores of 98.6 % on UCSD Ped2, 91.7 % on CUHK Avenue, and 83.7 % on ShanghaiTech. These results suggest that the proposed approach can generalize well across different scenes, perspectives, and types of anomalous behavior.