Adversarial erasure network based on multi-instance learning for weakly supervised video anomaly detection

Abstract

Weakly supervised video anomaly detection (WSVAD) aims to precisely locate temporal windows of abnormal events in untrimmed videos using only video-level labels. By accurately locating anomalies, WSVAD has great application potential in the security domain and contributes to the progress of smart city development. However, the lack of frame-level annotations during training makes it highly challenging to infer the status of each frame. Multiple-Instance Learning (MIL) is the dominant method in WSVAD. Due to the limitation of video-level annotations, most MIL-based methods detect obvious abnormal segments to represent the overall anomaly level of the video while overlooking weak abnormal segments. To focus on the discrimination of weak anomalies, we propose a novel WSVAD framework named Adversarial Erasure Network (AE-Net). AE-Net consists of two key components: (1) a dual-branch architecture that highlights weak anomalies by erasing the most obvious abnormal features and combining the erased features with the original ones. (2) a novel triplet loss function that improves weak anomaly representation by separating abnormal and normal features in the erased feature space. Through the above design, AE-Net can reduce false negatives in real-world anomaly detection. Extensive experiments on three WSVAD benchmarks demonstrate that our method outperforms most existing state-of-the-art methods. Specifically, AE-Net achieves an AUC of 88.40% on the UCF-Crime dataset and 98.27% on the ShanghaiTech dataset, which demonstrates that AE-Net can effectively distinguish between normal and abnormal events. Moreover, AE-Net achieves an AP of 85.13% on the XD-Violence dataset, which highlights that AE-Net can accurately detect abnormal events.