Using Visible+NIR information for CNN face recognition

Abstract

Basically every digital camera can acquire information that extends the visible spectrum since the sensor is sensitive to the near-infrared spectrum - and this sometimes requires only minor modifications to the device. So, using a conventional digital camera and stripping off the internal ICF (Infrared Cut-off Filter) filter, we use the captured Visible + NIR images (also called full-spectrum or VNIR images) for the classical face recognition problem. The camera stores the image as 3 channels RGB files, and training and evaluating a CNN with these full-spectrum images lead to surprisingly good results. On the contrary, doing the same with RGB+NIR (4 channels) does not perform as well. The paper shows that the contribution of the blue channel to this task is weak, and the recognition rate raises significantly when NIR is added to the channels, adding information and increasing signal to noise ratio especially in the blue channel.