Elevating urban surveillance: A deep CCTV monitoring system for detection of anomalous events via human action recognition

In the face of urbanization and the widespread use of CCTV cameras, the processing of surveillance videos has gained importance. This study endeavors to create a city-wide monitoring system utilizing human action recognition that can elevate the social sustainability of citizens. The primary goal is to develop an entire framework to detect unusual events within urban environments, with a specific focus on identifying four aberrant actions: “falling,” “violence,” “loitering,” and “intrusion.”. The processing of CCTV images is vulnerable to adverse weather conditions, particularly impacting human detection and tracking when obstructions like body parts occlusion, such as during falling events. To address these challenges, the paper proposes tracking compensation techniques that boost the system’s ability to detect anomalies without requiring additional training. The proposed approach demonstrates a remarkable 21.21% enhancement in detecting falling events, without compromising its handling of other event types. Overall, the system achieves an impressive average F1 score of 93% across diverse event categories. The system’s effectiveness is thoroughly assessed through an extensive subway domain case study, shedding light on its robustness and adaptability for potential real-world deployment. This study also delves into transfer learning dynamics based on sample quantity and pre-training with relevant human-of-interest data.