A Robust Method for Electrical Equipment Infrared and Visible Image Registration

The integration of infrared and visible images can take advantage of temperature information from infrared modality and sharp appearance from visible modality, and therefore it is helpful to improve the accuracy of localization and fault diagnosis of electrical equipment. A key step towards integration analysis is to register the images in infrared and visible modalities. In this paper, we propose a new method for infrared and visible image registration. In order to deal with large difference between these two modalities, we first transform both infrared and visible images into radiation-invariant maps. Then, LoFTR, which is a self-attention based deep neural network, is adopted to extract and match features based on the radiation-invariant maps. Finally, we utilize a progressive sample consensus (PROSAC) algorithm to estimate the transformation parameters, based on which the infrared image can be transformed into the corresponding visible image coordinates. Experiments on an electrical equipment dataset show that our proposed method is robust to both radiation and geometric variations.