SLRCNN: Integrating sparse and low-rank with a CNN denoiser for hyperspectral and multispectral image fusion

IF 7.6 Q1 REMOTE SENSING
{"title":"SLRCNN: Integrating sparse and low-rank with a CNN denoiser for hyperspectral and multispectral image fusion","authors":"","doi":"10.1016/j.jag.2024.104227","DOIUrl":null,"url":null,"abstract":"<div><div>Fusion of hyperspectral image (HSI) and multispectral image (MSI) is a prevalent scheme to generate a HSI with enhanced spatial resolution. Current methods often fail to sufficiently leverage the effective spectral and spatial priors existing in the observed HSI and MSI to further enhance the fusion performance. To address this limitation, this paper proposes a novel HSI-MSI fusion approach, which integrates Sparse and Low Rank with a CNN denoiser (SLRCNN) while considering spectral dictionary optimization. Firstly, an initialized spectral dictionary is derived from the HSI. Next, the spatial coefficients optimization model is established by incorporating the sparse prior, local low-rank prior, and plugged image prior simultaneously, where the <span><math><mrow><mspace></mspace><msub><mi>l</mi><mn>1</mn></msub></mrow></math></span> norm is imposed to promote the sparse prior, and the super-pixel segmentation strategy is conducted on the MSI to impose the local low-rank prior while a well-trained CNN denoiser is plugged in to enforce the image prior. Then, the spectral dictionary optimization model is constructed to refine the initial spectral dictionary, capturing more detailed spectral characteristics to further improve the fusion results. Finally, the optimization process involves applying the split-augmented Lagrangian shrinkage method and the alternating direction method of multipliers. Experimental results on simulated and real datasets, namely the Pavia University dataset, the Indian Pines dataset, and the EO-1 dataset, indicate that SLRCNN outperforms existing state-of-the-art approaches at 4x, 5x, and 6x resolutions in both qualitative and quantitative evaluation results. Specifically, the peak signal-to-noise ratio (PSNR) of SLRCNN is improved by more than 0.9 dB, 0.9 dB, and 0.2 dB while the spectral angle mapper (SAM) is decreased by more than 0.1, 0.2, and 0.2 in degree compared to other state-of-the-art methods across three datasets, respectively, which underscores the effectiveness of SLRCNN in leveraging both spatial detail reconstruction and spectral preservation.</div></div>","PeriodicalId":73423,"journal":{"name":"International journal of applied earth observation and geoinformation : ITC journal","volume":null,"pages":null},"PeriodicalIF":7.6000,"publicationDate":"2024-10-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"International journal of applied earth observation and geoinformation : ITC journal","FirstCategoryId":"1085","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1569843224005831","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"REMOTE SENSING","Score":null,"Total":0}
引用次数: 0

Abstract

Fusion of hyperspectral image (HSI) and multispectral image (MSI) is a prevalent scheme to generate a HSI with enhanced spatial resolution. Current methods often fail to sufficiently leverage the effective spectral and spatial priors existing in the observed HSI and MSI to further enhance the fusion performance. To address this limitation, this paper proposes a novel HSI-MSI fusion approach, which integrates Sparse and Low Rank with a CNN denoiser (SLRCNN) while considering spectral dictionary optimization. Firstly, an initialized spectral dictionary is derived from the HSI. Next, the spatial coefficients optimization model is established by incorporating the sparse prior, local low-rank prior, and plugged image prior simultaneously, where the l1 norm is imposed to promote the sparse prior, and the super-pixel segmentation strategy is conducted on the MSI to impose the local low-rank prior while a well-trained CNN denoiser is plugged in to enforce the image prior. Then, the spectral dictionary optimization model is constructed to refine the initial spectral dictionary, capturing more detailed spectral characteristics to further improve the fusion results. Finally, the optimization process involves applying the split-augmented Lagrangian shrinkage method and the alternating direction method of multipliers. Experimental results on simulated and real datasets, namely the Pavia University dataset, the Indian Pines dataset, and the EO-1 dataset, indicate that SLRCNN outperforms existing state-of-the-art approaches at 4x, 5x, and 6x resolutions in both qualitative and quantitative evaluation results. Specifically, the peak signal-to-noise ratio (PSNR) of SLRCNN is improved by more than 0.9 dB, 0.9 dB, and 0.2 dB while the spectral angle mapper (SAM) is decreased by more than 0.1, 0.2, and 0.2 in degree compared to other state-of-the-art methods across three datasets, respectively, which underscores the effectiveness of SLRCNN in leveraging both spatial detail reconstruction and spectral preservation.
SLRCNN:将稀疏和低秩与 CNN 去噪器相结合,用于高光谱和多光谱图像融合
高光谱图像(HSI)与多光谱图像(MSI)的融合是生成空间分辨率更高的 HSI 的普遍方案。目前的方法往往不能充分利用观测到的高光谱图像和多光谱图像中存在的有效光谱和空间先验来进一步提高融合性能。为解决这一局限性,本文提出了一种新颖的 HSI-MSI 融合方法,该方法将稀疏低等级与 CNN 去噪器(SLRCNN)相结合,同时考虑了光谱字典优化。首先,从 HSI 导出初始化光谱字典。接下来,同时结合稀疏先验、局部低秩先验和插入式图像先验,建立空间系数优化模型,其中施加 l1 准则以促进稀疏先验,在 MSI 上执行超像素分割策略以施加局部低秩先验,同时插入训练有素的 CNN 去噪器以执行图像先验。然后,构建光谱字典优化模型以完善初始光谱字典,捕捉更详细的光谱特征,从而进一步改善融合结果。最后,优化过程包括应用分裂增量拉格朗日收缩法和乘数交替方向法。在模拟和真实数据集(即帕维亚大学数据集、印度松树数据集和 EO-1 数据集)上的实验结果表明,SLRCNN 在 4x、5x 和 6x 分辨率下的定性和定量评估结果均优于现有的先进方法。具体来说,在三个数据集上,SLRCNN 的峰值信噪比 (PSNR) 分别提高了 0.9 dB、0.9 dB 和 0.2 dB 以上,而光谱角映射器 (SAM) 与其他先进方法相比分别降低了 0.1、0.2 和 0.2 度以上,这凸显了 SLRCNN 在利用空间细节重建和光谱保护方面的有效性。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
International journal of applied earth observation and geoinformation : ITC journal
International journal of applied earth observation and geoinformation : ITC journal Global and Planetary Change, Management, Monitoring, Policy and Law, Earth-Surface Processes, Computers in Earth Sciences
CiteScore
12.00
自引率
0.00%
发文量
0
审稿时长
77 days
期刊介绍: The International Journal of Applied Earth Observation and Geoinformation publishes original papers that utilize earth observation data for natural resource and environmental inventory and management. These data primarily originate from remote sensing platforms, including satellites and aircraft, supplemented by surface and subsurface measurements. Addressing natural resources such as forests, agricultural land, soils, and water, as well as environmental concerns like biodiversity, land degradation, and hazards, the journal explores conceptual and data-driven approaches. It covers geoinformation themes like capturing, databasing, visualization, interpretation, data quality, and spatial uncertainty.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信