LKAT-GAN: A GAN for Thermal Infrared Image Colorization Based on Large Kernel and AttentionUNet-Transformer

Abstract

Because thermal infrared (TIR) images are not affected by light and foggy environments, which are widely used in various night traffic scenarios. Especially, thermal infrared images also play an important role in autonomous vehicles. However, low contrast and lack of chromaticity have always been their problems. Image colorization is a vital technique to improve the quality of TIR images, which is beneficial to human interpretation and downstream tasks. Despite thermal infrared image colorization methods have been rapidly improved, the detail blurriness and color distortion in colorized images remain under-addressed. Mostly because these methods cannot effectively extract the ambiguous feature information of TIR images. Hence, we propose a large kernel (LK) U-Net and Attention_U-Net-Transformer (ViT-Based) based generative adversarial network. An LK_U-Net is designed to extract the feature of TIR images. Then, a branch structure composed of Attention_U-Net and ViT-Based can provide the network with semantic information from different perspectives to decode features. In addition, a composite loss function is employed to ensure the network generates a high-quality colorized image. The proposed method is evaluated on KAIST and IRVI datasets. Experimental results demonstrate the superiority of the proposed LKAT-GAN over other methods for the task of thermal infrared image colorization. The code is available at https://github.com/jinxinhuo/LKAT-GAN .