Deformable part models with CNN features for facial landmark detection under occlusion

Abstract

Detecting and localizing facial regions in images is a fundamental building block of many applications in the field of affective computing and human-computer interaction. This allows systems to do a variety of higher level analysis such as facial expression recognition. Facial expression recognition is based on the effective extraction of relevant facial features. Many techniques have been proposed to deal with the robust extraction of these features under a wide variety of poses and occlusion conditions. These techniques include Deformable Part Models (DPM's), and more recently deep Convolutional neural networks (CNN's). Recently, hybrid models based on DPMs and CNNs have been proposed considering the generalization properties of CNNs and DPMs. In this work we propose a combined system, using CNN's as features for a DPM with a focus on dealing with occlusion. We also propose a method of face detection allowing occluded regions to be detected and explicitly ignored during the detection step. The resulting system is quite robust to a wide variety of occlusions achieving accuracies comparable to that of other state of the art systems.