Advanced deep learning for masked individual surveillance

Abstract

During Covid-19 pandemic, face masks have become a ubiquitous protective measure. This poses new challenges for surveillance systems that heavily rely on facial recognition. To address this critical issue, we present a novel enhanced surveillance system that leverages deep learning techniques to tackle two crucial tasks simultaneously: anomaly detection of masked individuals' activities and masked face completion for accurate recognition. For anomaly detection, we employ a custom-designed deep neural network capable of processing real-time video streams. Finding a dataset of anomaly events of masked individuals is a big challenge for us. We handle this challenge using efficient techniques such as Dlib library and other image processing techniques. The network is trained on a diverse dataset encompassing normal and abnormal activities of masked individuals, enabling it to identify suspicious behaviors effectively. The surveillance cameras will exchange information, using a suitable network protocol, about detected anomalies and share relevant image data to aid in decision-making and choose the best images for further processing. In the context of masked face completion, we present a novel architecture called CCGAN network that is a combination of convolutional neural network (CNN) and conditioned generative adversarial network (CGAN) to generate the hidden parts of the face in a form that is accurate and close to the original face shape, as shown in our results. We conduct extensive experiments on publicly available datasets, demonstrating superior performance in both anomaly detection and masked face completion tasks. We have achieved 90% accuracy for anomaly detection of masked people.