PIFRNet: A progressive infrared feature-refinement network for single infrared image super-resolution

Abstract

Infrared (IR) image super-resolution is essential to enrich the spatial information of low-resolution IR images for facilitating computer vision tasks in low-illumination environments. However, IR images lack the textural details and colour information. Hence, existing super-resolution methods for visible images generally yield unsatisfactory super-resolution performance for IR images. Moreover, most super-resolution methods tailored for IR images neglect their intrinsic characteristics, thereby limiting their performance. Thus, this study proposes a progressive IR feature-refinement network for single IR image super-resolution (PIFRNet). First, a sequence of cascade IR feature-refinement blocks is designed in the deep feature-extraction path to extract and refine the deep features of IR images progressively. Subsequently, hybrid CNN–Transformer modules are developed to further enhance the long-range dependency attention of the extracted features. Then, a bespoke feature-fusion strategy with down connections, skip connections, and efficient feature-fusion blocks is implemented to effectively integrate low- to high-level IR image deep features. Finally, a dedicated large-scale super-resolution dataset of IR images is constructed for training and testing the IR image super-resolution networks. Extensive ablation studies and comparative experiments are conducted on this dataset. The ablation study results verify that the components and strategies designed for the proposed network are suitable for IR image super-resolution. The comparative experimental results demonstrate the superiority of the proposed network over other state-of-the-art visible/IR image super-resolution networks. Furthermore, a robustness test and an object-detection experiment are performed to prove the adaptability of the proposed network to various types of IR images and the significant object detection accuracy improvement achieved using the proposed PIFRNet.