A Robust Infrared and Visible Image Registration Method for Dual-Sensor UAV System

Abstract

Single-modal image registration methods are generally not feasible for visible and infrared images. Besides, multimodal image registration methods still suffer from uneven distribution of extracted features, low repeatability, and ambiguous features. To address these issues, a coarse-to-fine infrared and visible image registration approach for a dual-sensor unmanned aerial vehicle (UAV) imaging system is proposed, which is resilient to the difference in focal lengths and field of view. First, in the coarse registration step, the infrared image is transformed to the same scale as the visible image by using the similarity transformation. This operation makes the proposed method robust to the variation of the field of view. Then, the feature point pairs are initialized using feature detectors in the infrared image’s blocked phase congruency feature map. Next, the feature point pairs are optimized by estimating the offset based on the relationship between the constructed feature descriptors. Finally, using elastic deformation, the pixel-level registered infrared image is obtained. Extensive experiments demonstrate the superior performance of the proposed coarse-to-fine image registration methodology in real infrared-visible image pairs. The code and dataset are available at https://drive.google.com/drive/folders/1mpUWwHUbKTrBdOrNMNRRnuJclDUAC7nU?usp=sharing .