Face Recognition Despite Wearing Glasses

In this paper we address the challenge of performing face recognition on human faces that are wearing glasses. This is a common problem for face recognition and automatic identity checking at airports, as passengers frequently forget to remove their glasses when passing through customs. In order to solve this problem, we first propose an automatic glasses presence detection model based on the tree-pictorial-structured face detection model and such model can detect the presence of glasses and further assign landmarks on the rim, hinge, and bridge of the glasses on frontal faces. Experimental results show that the glasses detection rate is highly satisfactory for various face databases. Secondly, based on the landmarks on glasses, we apply the non-local colour total variation (CTV) inpainting approach in an attempt to remove the glasses; also, we apply the deep learning technique to further remove the traces of glasses and light reflection on lenses by regarding them as noises. Finally, experiments for face recognition after glasses removal are conducted by using some typical approaches and the results show that our glasses removal framework can improve face recognition accuracy significantly.