Hybrid sensing face detection and recognition

Abstract

The capability to track, detect, and identify human targets in highly cluttered scenes under extreme conditions, such as in complete darkness or in battlefield, has been one of the primary tactical advantages in military operations. In this paper, we propose a new collaborative, multi-spectrum sensing solution to achieve face detection and registration under low lighting conditions. We construct a novel type of hybrid sensors by combining a pair of near infrared (NIR) cameras and a thermal camera (a long wave infrared LWIR camera). We strategically surround each NIR sensor with a ring of LED IR flashes in order to capture the “red-eye”, or more precisely, the “bright-eye” effect of the target. The bright-eyes are used to localize the 3D position of eyes and face. The recovered 3D information can be further used to warp the thermal face imagery to frontal-parallel pose so that additional tasks such as face recognition can be reliably conducted, especially with the assistance of accurate eye locations.