Qian Jiang , Qianqian Wang , Shengfa Miao , Xin Jin , Shin-Jye Lee , Michal Wozniak , Shaowen Yao
{"title":"SR_ColorNet: Multi-path attention aggregated and mask enhanced network for the super resolution and colorization of panchromatic image","authors":"Qian Jiang , Qianqian Wang , Shengfa Miao , Xin Jin , Shin-Jye Lee , Michal Wozniak , Shaowen Yao","doi":"10.1016/j.eswa.2025.127091","DOIUrl":null,"url":null,"abstract":"<div><div>Improving the spatial and spectral resolution of remote sensing images is important in areas such as environmental monitoring and military reconnaissance. High-quality remote sensing images with high spatial and spectral resolution often yield better results in these fields. However, a single sensor cannot obtain high spatial resolution color remote sensing images and can only obtain grayscale panchromatic (PAN) images and low spatial resolution multispectral(MS) images. In existing methods, obtaining high spatial resolution color images is tough when only PAN images are input. Image super-resolution (SR) models can improve the spatial resolution of the image, but not the spectral resolution. Image colorization models can improve spectral resolution, not spatial resolution. Pan-sharpening models depend on paired PAN and MS images. This study aggregates SR and colorization tasks of PAN images in the same model and completed simultaneously. We propose a multi-path network (SR_ColorNet) for recovering PAN image resolution, utilizing both Transformer and Convolutional Neural Network (CNN) architectures. Our method includes three key stages: shallow feature extraction, deep feature extraction, and feature reconstruction. Shallow feature extraction employs a VGG19 for multi-path feature extraction. The deep feature extraction stage consists of three modules: the SR transformer (SRT) module for recovering spatial information, ECA Channel Mixing Block (ECMB) for retaining and transmitting significant feature information, and Fusion Feature Processing Block (FFPB) for processing information. In the feature reconstruction stage, a Masked Feature Enhancement (MFE) module is proposed to enhance the feature. Our SR_ColorNet performed well at image SR and colorization in experiments, according to objective metrics and visual quality. Our code is available at <span><span>https://github.com/QianqianWang1325/SR_ColorNet_main</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":50461,"journal":{"name":"Expert Systems with Applications","volume":"276 ","pages":"Article 127091"},"PeriodicalIF":7.5000,"publicationDate":"2025-03-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Expert Systems with Applications","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0957417425007134","RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE","Score":null,"Total":0}
引用次数: 0
摘要
提高遥感图像的空间和光谱分辨率在环境监测和军事侦察等领域非常重要。在这些领域,具有高空间和光谱分辨率的高质量遥感图像往往能产生更好的效果。然而,单个传感器无法获得高空间分辨率的彩色遥感图像,只能获得灰度全色(PAN)图像和低空间分辨率的多光谱(MS)图像。在现有方法中,如果只输入 PAN 图像,则很难获得高空间分辨率的彩色图像。图像超分辨率(SR)模型可以提高图像的空间分辨率,但不能提高光谱分辨率。图像着色模型可以提高光谱分辨率,但不能提高空间分辨率。平移锐化模型取决于成对的 PAN 和 MS 图像。本研究将 PAN 图像的 SR 和着色任务整合到同一模型中,并同时完成。我们提出了一种利用变换器和卷积神经网络(CNN)架构恢复 PAN 图像分辨率的多路径网络(SR_ColorNet)。我们的方法包括三个关键阶段:浅层特征提取、深层特征提取和特征重建。浅层特征提取采用 VGG19 进行多路径特征提取。深度特征提取阶段由三个模块组成:用于恢复空间信息的 SR 变换器(SRT)模块、用于保留和传输重要特征信息的 ECA 通道混合块(ECMB)以及用于处理信息的融合特征处理块(FFPB)。在特征重建阶段,提出了一个掩蔽特征增强(MFE)模块来增强特征。在实验中,根据客观指标和视觉质量,我们的 SR_ColorNet 在图像 SR 和色彩化方面表现出色。我们的代码见 https://github.com/QianqianWang1325/SR_ColorNet_main。
SR_ColorNet: Multi-path attention aggregated and mask enhanced network for the super resolution and colorization of panchromatic image
Improving the spatial and spectral resolution of remote sensing images is important in areas such as environmental monitoring and military reconnaissance. High-quality remote sensing images with high spatial and spectral resolution often yield better results in these fields. However, a single sensor cannot obtain high spatial resolution color remote sensing images and can only obtain grayscale panchromatic (PAN) images and low spatial resolution multispectral(MS) images. In existing methods, obtaining high spatial resolution color images is tough when only PAN images are input. Image super-resolution (SR) models can improve the spatial resolution of the image, but not the spectral resolution. Image colorization models can improve spectral resolution, not spatial resolution. Pan-sharpening models depend on paired PAN and MS images. This study aggregates SR and colorization tasks of PAN images in the same model and completed simultaneously. We propose a multi-path network (SR_ColorNet) for recovering PAN image resolution, utilizing both Transformer and Convolutional Neural Network (CNN) architectures. Our method includes three key stages: shallow feature extraction, deep feature extraction, and feature reconstruction. Shallow feature extraction employs a VGG19 for multi-path feature extraction. The deep feature extraction stage consists of three modules: the SR transformer (SRT) module for recovering spatial information, ECA Channel Mixing Block (ECMB) for retaining and transmitting significant feature information, and Fusion Feature Processing Block (FFPB) for processing information. In the feature reconstruction stage, a Masked Feature Enhancement (MFE) module is proposed to enhance the feature. Our SR_ColorNet performed well at image SR and colorization in experiments, according to objective metrics and visual quality. Our code is available at https://github.com/QianqianWang1325/SR_ColorNet_main.
期刊介绍:
Expert Systems With Applications is an international journal dedicated to the exchange of information on expert and intelligent systems used globally in industry, government, and universities. The journal emphasizes original papers covering the design, development, testing, implementation, and management of these systems, offering practical guidelines. It spans various sectors such as finance, engineering, marketing, law, project management, information management, medicine, and more. The journal also welcomes papers on multi-agent systems, knowledge management, neural networks, knowledge discovery, data mining, and other related areas, excluding applications to military/defense systems.