Crowd dynamics analysis and behavior recognition in surveillance videos based on deep learning

Abstract

Video surveillance is widely adopted across various sectors for purposes such as law enforcement, COVID-19 isolation monitoring, and analyzing crowds for potential threats like flash mobs or violence. The vast amount of data generated daily from surveillance devices holds significant potential but requires effective analysis to extract value. Detecting anomalous crowd behavior, which can lead to chaos and casualties, is particularly challenging in video surveillance due to its labor-intensive nature and susceptibility to errors. To address these challenges, this research contributes in two key areas: first, by creating a diverse and representative video dataset that accurately reflects real-world crowd dynamics across eight different categories; second, by developing a reliable framework, ‘CRAB-NET,’ for automated behavior recognition. Extensive experimentation and evaluation, using Convolutional Long Short-Term Memory networks (ConvLSTM) and Long-Term Recurrent Convolutional Networks (LRCN), validated the effectiveness of the proposed approach in accurately categorizing behaviors observed in surveillance videos. The employed models were able to achieve the accuracy score of 99.46% for celebratory crowd, 99.98% for formal crowd and 96.69% for violent crowd. The demonstrated accuracy of 97.20% for comprehensive dataset achieved by the LRCN underscores its potential to revolutionize crowd behavior analysis. It ensures safer mass gatherings and more effective security interventions. Incorporating AI-powered crowd behavior recognition like ‘CRAB-NET’ into security measures not only safeguards public gatherings but also paves the way for proactive event management and predictive safety strategies.