Real-time masked face recognition and authentication with convolutional neural networks on the web application

Abstract

The COVID-19 outbreak has highlighted the importance of wearing a face mask to prevent virus transmission. During the peak of the pandemic, everyone was required to wear a face mask both inside and outside the building. Nowadays, even though the pandemic has passed, it is still necessary to wear a face mask in some situations/areas. Nevertheless, a face mask becomes a major barrier, especially in places where full-face authentication is required; most facial recognition systems are unable to recognize masked faces accurately, thereby resulting in incorrect predictions. To address this challenge, this study proposes a web-based application system to accomplish three main tasks: (1) recognizing, in real-time, whether an individual entering the location is wearing a face mask; and (2) correctly identifying an individual as a biometric authentication despite facial features obscured by a face mask with varying types, shapes and colors. (3) easily updating the recognition model with the most recent user list, with a user-friendly interface from the real-time web application. The underlying model to perform detection and recognition is convolutional neural networks. In this study, we experimented with VGG16, VGGFace, and InceptionResNetV2. Experimental cases to determine model performance are; using only masked-face images, and using both full-face and masked-face images together. We evaluate the models using performance metrics including accuracy, recall, precision, F1-score, and training time. The results have shown superior performance compared with those from related works. Our best model could reach an accuracy of 93.3%, a recall of 93.8%, and approximately 93-94% for precision and F1-score, when recognizing 50 individuals.