SV-CGAN: Infrared image generation based on CycleGAN

Abstract

Infrared image generation holds significant application value in fields such as night vision surveillance, military reconnaissance, autonomous driving, and disaster rescue. It overcomes the imaging limitations of visible light sensors under low-light conditions, harsh weather, or complex environments, providing critical data support for all-weather perception and decision-making. However, existing deep learning-based methods for generating infrared images still face challenges, including imprecise cross-modal feature alignment and modeling bias in thermal radiation distribution, which result in generated images with blurred details, artifact noise, and weak generalization across different spectral bands, severely limiting their practical applicability. This paper proposes the SV-CGAN model, which integrates semantic segmentation with CycleGAN and employs a generator combining U-Net and an improved Vision Transformer (ViT) at the bottleneck, along with an optimized multi-task loss function. This approach enables the generation of high-quality infrared images under conditions of unpaired data. Experimental results demonstrate that SV-CGAN achieves a peak signal-to-noise ratio (PSNR) of 30.8315 dB and a structural similarity index (SSIM) of 0.8934 on the Multispectral Pedestrian Dataset (MPD), outperforming CycleGAN (PSNR 26.5260 dB, SSIM 0.8257) by 16.2 % and 8.2 %, outperforming U-GAT-IT by 4.0 % and 2.7 %, respectively. Additionally, the Fréchet Inception Distance (FID) metric decreased by 33.1 % from 114.9876 to 76.9776. The model effectively achieves visible-to-infrared image translation under unpaired training conditions, producing images with higher realism and detail retention.