Perceptive spectral transformer unfolding network with multiscale mixed training for arbitrary-scale hyperspectral and multispectral image fusion

Hyperspectral imaging offers rich spectral information but often suffers from a tradeoff between spatial and spectral resolutions owing to hardware limitations. To address this, hyperspectral image (HSI)-multispectral image (MSI) fusion techniques have emerged, which combines low-resolution HSI (LR-HSI) with high-resolution MSI (HR-MSI) to generate HR-HSI. However, existing methods often struggle with generalization across varying image resolutions and lack interpretability due to reliance on deep learning models without physical degradation constraints. This study introduces two major innovations to overcome these challenges: (1) a resolution-independent unfolding algorithm and a multiscale training framework, which allows flexible adaptation to LR-HSI of any resolution without increasing model complexity, thereby enhancing generalization in dynamic remote sensing environments and (2) a novel degradation design using real LR-HSI and HR-MSI as priors to guide spatial-spectral degradation and, in the external framework, implementing degradation constraints, thereby ensuring accurate approximation of true degradation processes. In addition, this study proposes a perceptive spectral transformer with perceptive spectral attention in a U-Net architecture to adaptively transfer spectral information, improving fusion accuracy. Experimental results highlight advantages of our approach under both single-scale training and multiscale mixed training conditions. Compared with eight state-of-the-art fusion algorithms, our approach demonstrates exceptional performance under single-scale training conditions. More importantly, through multiscale mixed training, its performance is further enhanced, achieving super-resolution magnifications ranging from 4 × to 128 ×, validating the effectiveness of the framework. Experiments demonstrated the robustness and practical applicability of the proposed approach in simulated and real-world scenarios; the code is available at https://github.com/XWangBin/PSTUN.