Lucas-Kanade-based Face Detection Network: An Unsupervised Approach to Improve the Precision and Stability of video-based Face Detector

Abstract

Though significant works have been made in image-based face detection, precise and stable video-based face detection remains a big challenge. What's more, the jittering of bounding boxes has an important impact on the stability of face attributes analysis. To address these issues, this paper presents an unsupervised single-stage face detector, named Lucas-Kanade-based face detection network (LK-FDN). Our key idea is that the detections of the same landmark in adjacent frames should be coherent with optical flow registration. By combining RetinaFace and the optical flow algorithm in a multi-task learning framework, LK-FDN does not require any video-level annotation, because the coherency of optical flow is a source of supervision and can be leveraged during detector training. Essentially, LK-FDN augments the training loss function with a registration loss. Supervised by the registration loss, the retraining process is conducted, which computes optical flow registration in the forward pass, and back-propagates gradients that ensure temporal coherency in the detector. The output of LK-FDN is a more precise and stable video-based face detector. Specifically, the advantages of LK-FDN are summarized in the following three aspects: (1) Compared with RetinaFace(baseline), LK-FDN improves the average precision (AP) by 0.43%, 0.23%, and 4.1% respectively on the easy, medium, and hard group of WIDER FACE. (2) Compared with RetinaFace, LK-FDN improves the precision without spending any extra inference time in test time, because LK is conducted during the retraining backward process. (3) LK-FDN significantly shrinks the jittering in video detection on the self-built ElevatorFace dataset without any video-level annotation. (4) According to the self-built evaluation criterion, the score of stability is improved from 88.409 to 97.119, which verifies the effectiveness of the LK-FDN.