CA-VAD: Caption Aware Video Anomaly Detection in surveillance videos

In video anomaly detection, identifying abnormal events using weakly supervised video-level labels is often tackled with multiple instance learning (MIL). However, traditional methods struggle to capture temporal relationships between segments and extract discriminative features for distinguishing normal from anomalous events. To address these challenges, we propose Caption Aware Video Anomaly Detection (CA-VAD), a framework that integrates visual and textual features for enhanced semantic understanding of scenes. Unlike conventional approaches relying solely on visual data, CA-VAD uses a pre-trained video captioning model to generate textual descriptions, transforming them into semantic embeddings that enrich visual features. These textual cues improve the differentiation between normal and abnormal events. CA-VAD incorporates an Attention-based Multi-Scale Temporal Network (A-MTN) to process visual and textual inputs, capturing temporal dynamics effectively. Experiments on CUHK Avenue, ShanghaiTech, UCSD Ped2, and XD-Violence datasets show that CA-VAD outperforms state-of-the-art methods, achieving superior accuracy and robustness.