A Compact Tri-Modal Camera Unit for RGBDT Vision

Abstract

The combination of RGBD and thermal cameras in multi-modal person-centric vision applications has great potential. As a complementary modality, thermal cameras can compensate for weaknesses such as the inability to operate in absolute darkness of conventional RGB cameras or the range limitations associated with consumer depth cameras, resulting in a more robust computer vision system. In addition, the high contrast between persons and their surroundings in thermal images can ease fundamental detection and segmentation tasks. Unfortunately, the market supply of low-cost consumer RGBDT vision systems is non-existent at the moment, which slows down progress in the field of person-centric vision. We address this problem by proposing a Compact Tri-modal CAmera uniT (CTCAT) for RGBDT vision, which can be manufactured from off-the-shelf components and 3D printed parts. CTCAT features a 1280 × 720 RGB camera, a 640 × 480 structured light depth camera with an operating range of 0.6 − 8m, and a 160 × 120 uncooled radiometric thermal camera. RGB, depth, and thermal images can be captured simultaneously at frame rates up to 9 fps. In this work, we describe the components, fabrication, and calibration of CTCAT. In addition, a new multi-modal calibration target suitable for the geometric calibration of RGB, depth, and thermal cameras is presented, which offers advantages over the state of the art in terms of contrast and practicality. Moreover, radiometric calibration of CTCAT is performed to evaluate the applicability to person-centric vision applications requiring radiometry.