UpAttTrans: Upscaled attention based transformer for facial image super-resolution

Abstract

Image super-resolution (SR) aims to reconstruct high-quality images from low-resolution inputs, a task particularly challenging in face-related applications due to extreme degradations and modality differences (e.g., visible, low-resolution, near-infrared). Conventional convolutional neural networks (CNNs) and GAN-based approaches have achieved notable success; however, they often struggle with preserving identity and fine structural details at high upscaling factors. In this work, we introduce UpAttTrans, a novel attention mechanism that connects original and upsampled features for better detail recovery based on vision transformer for SR. The core generator leverages a custom UpAttTrans module that translates input image patches into embeddings, processes them through transformer layers enhanced with connector-up attention, and reconstructs high-resolution outputs with improved detail retention. We evaluate our model on the CelebA dataset across multiple upscaling factors ($4\times$, $8\times$, $16\times$, $32\times$, and $64\times$). UpAttTrans achieves a 24.63% increase in PSNR, 21.56% in SSIM, and 19.61% reduction in FID for $4\times$ and $8\times$ SR, outperforming state-of-the-art baselines. Additionally, for higher magnification levels, our model maintains strong performance, with average gains of 6.20% in PSNR and 21.49% in SSIM, indicating its robustness in extreme SR settings. These findings suggest that UpAttTrans holds significant promise for real-world applications such as face recognition in surveillance, forensic image enhancement, and cross-spectral matching, where high-quality reconstruction from severely degraded inputs is critical.