restoredit:基于多模态顺序扩散变压器的可靠卫星图像时间序列重建

IF 11.4 1区 地球科学 Q1 ENVIRONMENTAL SCIENCES
Qidi Shu, Xiaolin Zhu, Shuai Xu, Yan Wang, Denghong Liu
{"title":"restoredit:基于多模态顺序扩散变压器的可靠卫星图像时间序列重建","authors":"Qidi Shu,&nbsp;Xiaolin Zhu,&nbsp;Shuai Xu,&nbsp;Yan Wang,&nbsp;Denghong Liu","doi":"10.1016/j.rse.2025.114872","DOIUrl":null,"url":null,"abstract":"<div><div>Repetitive optical observations from satellites are crucial for monitoring earth surface dynamics over time. However, optical satellite image time series is severely affected by frequent data gaps due to clouds and shadows. While synthetic aperture radar (SAR) provides cloud-penetrating capabilities to complement missing optical data, recent advancements in time series reconstruction have shifted focus from incorporating single SAR image to exploiting SAR time series. However, current methods still struggle for challenging scenarios like highly dynamic surface, persistent data gaps, and exhibit poor resilience to inaccurate cloud masks. In this research, we approach the time series reconstruction problem from the perspective of conditional generation. We propose a multimodal diffusion framework termed RESTORE-DiT, which firstly promotes the sequence-level optical-SAR fusion through a diffusion framework. Specifically, date-matched SAR time series provide under-cloud surface dynamics to guide the denoising process of cloudy areas, and date information is embedded to account for irregular observation intervals and periodic patterns. Extensive experiments on three regions have shown the proposed method achieves state-of-the-art performance. RESTORE-DiT outperforms comparison methods by 2.87 dB in PSNR and a 27.2 % reduction in RMSE on France site. SAR and date information together increase PSNR by 2.41 dB. The reconstructed optical image time series is verified to accurately reflect the crop growth condition and support for long-term vegetation observations. In addition, RESTORE-DiT can be easily extended to other conditional reconstruction or prediction tasks for arbitrary time series image data, thus facilitating spatiotemporal analysis research. The codes will be public available at: <span><span>https://github.com/SQD1/RESTORE-DiT</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":417,"journal":{"name":"Remote Sensing of Environment","volume":"328 ","pages":"Article 114872"},"PeriodicalIF":11.4000,"publicationDate":"2025-06-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"RESTORE-DiT: Reliable satellite image time series reconstruction by multimodal sequential diffusion transformer\",\"authors\":\"Qidi Shu,&nbsp;Xiaolin Zhu,&nbsp;Shuai Xu,&nbsp;Yan Wang,&nbsp;Denghong Liu\",\"doi\":\"10.1016/j.rse.2025.114872\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><div>Repetitive optical observations from satellites are crucial for monitoring earth surface dynamics over time. However, optical satellite image time series is severely affected by frequent data gaps due to clouds and shadows. While synthetic aperture radar (SAR) provides cloud-penetrating capabilities to complement missing optical data, recent advancements in time series reconstruction have shifted focus from incorporating single SAR image to exploiting SAR time series. However, current methods still struggle for challenging scenarios like highly dynamic surface, persistent data gaps, and exhibit poor resilience to inaccurate cloud masks. In this research, we approach the time series reconstruction problem from the perspective of conditional generation. We propose a multimodal diffusion framework termed RESTORE-DiT, which firstly promotes the sequence-level optical-SAR fusion through a diffusion framework. Specifically, date-matched SAR time series provide under-cloud surface dynamics to guide the denoising process of cloudy areas, and date information is embedded to account for irregular observation intervals and periodic patterns. Extensive experiments on three regions have shown the proposed method achieves state-of-the-art performance. RESTORE-DiT outperforms comparison methods by 2.87 dB in PSNR and a 27.2 % reduction in RMSE on France site. SAR and date information together increase PSNR by 2.41 dB. The reconstructed optical image time series is verified to accurately reflect the crop growth condition and support for long-term vegetation observations. In addition, RESTORE-DiT can be easily extended to other conditional reconstruction or prediction tasks for arbitrary time series image data, thus facilitating spatiotemporal analysis research. The codes will be public available at: <span><span>https://github.com/SQD1/RESTORE-DiT</span><svg><path></path></svg></span>.</div></div>\",\"PeriodicalId\":417,\"journal\":{\"name\":\"Remote Sensing of Environment\",\"volume\":\"328 \",\"pages\":\"Article 114872\"},\"PeriodicalIF\":11.4000,\"publicationDate\":\"2025-06-16\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Remote Sensing of Environment\",\"FirstCategoryId\":\"5\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S0034425725002767\",\"RegionNum\":1,\"RegionCategory\":\"地球科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"ENVIRONMENTAL SCIENCES\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Remote Sensing of Environment","FirstCategoryId":"5","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0034425725002767","RegionNum":1,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ENVIRONMENTAL SCIENCES","Score":null,"Total":0}
引用次数: 0

摘要

卫星的重复光学观测对于监测地球表面长期动态至关重要。然而,由于云层和阴影的影响,光学卫星图像时间序列受到频繁的数据缺口的严重影响。虽然合成孔径雷达(SAR)提供了穿透云层的能力,以补充缺失的光学数据,但最近时间序列重建的进展已将重点从合并单个SAR图像转移到利用SAR时间序列。然而,目前的方法仍然难以应对具有挑战性的场景,如高度动态的地表、持续的数据缺口,以及对不准确的云掩模的适应性差。在本研究中,我们从条件生成的角度来研究时间序列重构问题。我们提出了一个多模态扩散框架RESTORE-DiT,它首先通过扩散框架促进序列级光学sar融合。具体而言,日期匹配的SAR时间序列提供云下表面动态来指导多云区域的去噪过程,并且嵌入日期信息以解释不规则的观测间隔和周期性模式。在三个区域的大量实验表明,该方法达到了最先进的性能。RESTORE-DiT在法国站点的PSNR优于比较方法2.87 dB, RMSE降低27.2%。SAR和日期信息共同使PSNR提高了2.41 dB。验证了重建的光学影像时间序列能够准确反映作物生长状况,为长期植被观测提供支持。此外,RESTORE-DiT可以很容易地扩展到任意时间序列图像数据的其他条件重建或预测任务中,从而便于时空分析研究。这些代码将在https://github.com/SQD1/RESTORE-DiT上公开。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
RESTORE-DiT: Reliable satellite image time series reconstruction by multimodal sequential diffusion transformer
Repetitive optical observations from satellites are crucial for monitoring earth surface dynamics over time. However, optical satellite image time series is severely affected by frequent data gaps due to clouds and shadows. While synthetic aperture radar (SAR) provides cloud-penetrating capabilities to complement missing optical data, recent advancements in time series reconstruction have shifted focus from incorporating single SAR image to exploiting SAR time series. However, current methods still struggle for challenging scenarios like highly dynamic surface, persistent data gaps, and exhibit poor resilience to inaccurate cloud masks. In this research, we approach the time series reconstruction problem from the perspective of conditional generation. We propose a multimodal diffusion framework termed RESTORE-DiT, which firstly promotes the sequence-level optical-SAR fusion through a diffusion framework. Specifically, date-matched SAR time series provide under-cloud surface dynamics to guide the denoising process of cloudy areas, and date information is embedded to account for irregular observation intervals and periodic patterns. Extensive experiments on three regions have shown the proposed method achieves state-of-the-art performance. RESTORE-DiT outperforms comparison methods by 2.87 dB in PSNR and a 27.2 % reduction in RMSE on France site. SAR and date information together increase PSNR by 2.41 dB. The reconstructed optical image time series is verified to accurately reflect the crop growth condition and support for long-term vegetation observations. In addition, RESTORE-DiT can be easily extended to other conditional reconstruction or prediction tasks for arbitrary time series image data, thus facilitating spatiotemporal analysis research. The codes will be public available at: https://github.com/SQD1/RESTORE-DiT.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Remote Sensing of Environment
Remote Sensing of Environment 环境科学-成像科学与照相技术
CiteScore
25.10
自引率
8.90%
发文量
455
审稿时长
53 days
期刊介绍: Remote Sensing of Environment (RSE) serves the Earth observation community by disseminating results on the theory, science, applications, and technology that contribute to advancing the field of remote sensing. With a thoroughly interdisciplinary approach, RSE encompasses terrestrial, oceanic, and atmospheric sensing. The journal emphasizes biophysical and quantitative approaches to remote sensing at local to global scales, covering a diverse range of applications and techniques. RSE serves as a vital platform for the exchange of knowledge and advancements in the dynamic field of remote sensing.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信