RGB-Thermal cameras calibration based on Maximum Index Map

Abstract

Calibration across multiple sensors is crucial in computer vision tasks, such as image registration, 3D reconstruction, and target tracking. RGB-Thermal (RGB-T) cameras, widely used as a multi-sensor combination, traditionally depend on manual calibration methods to determine extrinsic parameters. However, this manual process is intricate and lacks the capability for online calibration during use. To overcome these challenges, an online calibration algorithm for RGB-T camera extrinsic parameters is proposed. The algorithm first addresses modal differences by matching feature point pairs using the Maximum Index Map (MIM) feature. These matched features are then used to calculate the extrinsic parameters through homography and epipolar constraints between images. Additionally, a convenient intrinsic calibration method is introduced, one that does not rely on extrinsic infrared light sources, thereby overcoming the limitations of standard calibration boards, which are often unsuitable for RGB-T cameras. Experimental results demonstrate that the proposed online extrinsic calibration algorithm significantly simplifies the calibration process compared to traditional methods, achieving accurate calibration with only a pair of images. The method achieves an average reprojection error of 0.677 pixels for RGB images and 0.635 pixels for thermal images, highlighting its precision. The method’s effectiveness is further validated by the precision in RGB-T image registration.