Qidi Shu, Xiaolin Zhu, Shuai Xu, Yan Wang, Denghong Liu
{"title":"restoredit:基于多模态顺序扩散变压器的可靠卫星图像时间序列重建","authors":"Qidi Shu, Xiaolin Zhu, Shuai Xu, Yan Wang, Denghong Liu","doi":"10.1016/j.rse.2025.114872","DOIUrl":null,"url":null,"abstract":"<div><div>Repetitive optical observations from satellites are crucial for monitoring earth surface dynamics over time. However, optical satellite image time series is severely affected by frequent data gaps due to clouds and shadows. While synthetic aperture radar (SAR) provides cloud-penetrating capabilities to complement missing optical data, recent advancements in time series reconstruction have shifted focus from incorporating single SAR image to exploiting SAR time series. However, current methods still struggle for challenging scenarios like highly dynamic surface, persistent data gaps, and exhibit poor resilience to inaccurate cloud masks. In this research, we approach the time series reconstruction problem from the perspective of conditional generation. We propose a multimodal diffusion framework termed RESTORE-DiT, which firstly promotes the sequence-level optical-SAR fusion through a diffusion framework. Specifically, date-matched SAR time series provide under-cloud surface dynamics to guide the denoising process of cloudy areas, and date information is embedded to account for irregular observation intervals and periodic patterns. Extensive experiments on three regions have shown the proposed method achieves state-of-the-art performance. RESTORE-DiT outperforms comparison methods by 2.87 dB in PSNR and a 27.2 % reduction in RMSE on France site. SAR and date information together increase PSNR by 2.41 dB. The reconstructed optical image time series is verified to accurately reflect the crop growth condition and support for long-term vegetation observations. In addition, RESTORE-DiT can be easily extended to other conditional reconstruction or prediction tasks for arbitrary time series image data, thus facilitating spatiotemporal analysis research. The codes will be public available at: <span><span>https://github.com/SQD1/RESTORE-DiT</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":417,"journal":{"name":"Remote Sensing of Environment","volume":"328 ","pages":"Article 114872"},"PeriodicalIF":11.4000,"publicationDate":"2025-06-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"RESTORE-DiT: Reliable satellite image time series reconstruction by multimodal sequential diffusion transformer\",\"authors\":\"Qidi Shu, Xiaolin Zhu, Shuai Xu, Yan Wang, Denghong Liu\",\"doi\":\"10.1016/j.rse.2025.114872\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<div><div>Repetitive optical observations from satellites are crucial for monitoring earth surface dynamics over time. However, optical satellite image time series is severely affected by frequent data gaps due to clouds and shadows. While synthetic aperture radar (SAR) provides cloud-penetrating capabilities to complement missing optical data, recent advancements in time series reconstruction have shifted focus from incorporating single SAR image to exploiting SAR time series. However, current methods still struggle for challenging scenarios like highly dynamic surface, persistent data gaps, and exhibit poor resilience to inaccurate cloud masks. In this research, we approach the time series reconstruction problem from the perspective of conditional generation. We propose a multimodal diffusion framework termed RESTORE-DiT, which firstly promotes the sequence-level optical-SAR fusion through a diffusion framework. Specifically, date-matched SAR time series provide under-cloud surface dynamics to guide the denoising process of cloudy areas, and date information is embedded to account for irregular observation intervals and periodic patterns. Extensive experiments on three regions have shown the proposed method achieves state-of-the-art performance. RESTORE-DiT outperforms comparison methods by 2.87 dB in PSNR and a 27.2 % reduction in RMSE on France site. SAR and date information together increase PSNR by 2.41 dB. The reconstructed optical image time series is verified to accurately reflect the crop growth condition and support for long-term vegetation observations. In addition, RESTORE-DiT can be easily extended to other conditional reconstruction or prediction tasks for arbitrary time series image data, thus facilitating spatiotemporal analysis research. The codes will be public available at: <span><span>https://github.com/SQD1/RESTORE-DiT</span><svg><path></path></svg></span>.</div></div>\",\"PeriodicalId\":417,\"journal\":{\"name\":\"Remote Sensing of Environment\",\"volume\":\"328 \",\"pages\":\"Article 114872\"},\"PeriodicalIF\":11.4000,\"publicationDate\":\"2025-06-16\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Remote Sensing of Environment\",\"FirstCategoryId\":\"5\",\"ListUrlMain\":\"https://www.sciencedirect.com/science/article/pii/S0034425725002767\",\"RegionNum\":1,\"RegionCategory\":\"地球科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"ENVIRONMENTAL SCIENCES\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Remote Sensing of Environment","FirstCategoryId":"5","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0034425725002767","RegionNum":1,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ENVIRONMENTAL SCIENCES","Score":null,"Total":0}
RESTORE-DiT: Reliable satellite image time series reconstruction by multimodal sequential diffusion transformer
Repetitive optical observations from satellites are crucial for monitoring earth surface dynamics over time. However, optical satellite image time series is severely affected by frequent data gaps due to clouds and shadows. While synthetic aperture radar (SAR) provides cloud-penetrating capabilities to complement missing optical data, recent advancements in time series reconstruction have shifted focus from incorporating single SAR image to exploiting SAR time series. However, current methods still struggle for challenging scenarios like highly dynamic surface, persistent data gaps, and exhibit poor resilience to inaccurate cloud masks. In this research, we approach the time series reconstruction problem from the perspective of conditional generation. We propose a multimodal diffusion framework termed RESTORE-DiT, which firstly promotes the sequence-level optical-SAR fusion through a diffusion framework. Specifically, date-matched SAR time series provide under-cloud surface dynamics to guide the denoising process of cloudy areas, and date information is embedded to account for irregular observation intervals and periodic patterns. Extensive experiments on three regions have shown the proposed method achieves state-of-the-art performance. RESTORE-DiT outperforms comparison methods by 2.87 dB in PSNR and a 27.2 % reduction in RMSE on France site. SAR and date information together increase PSNR by 2.41 dB. The reconstructed optical image time series is verified to accurately reflect the crop growth condition and support for long-term vegetation observations. In addition, RESTORE-DiT can be easily extended to other conditional reconstruction or prediction tasks for arbitrary time series image data, thus facilitating spatiotemporal analysis research. The codes will be public available at: https://github.com/SQD1/RESTORE-DiT.
期刊介绍:
Remote Sensing of Environment (RSE) serves the Earth observation community by disseminating results on the theory, science, applications, and technology that contribute to advancing the field of remote sensing. With a thoroughly interdisciplinary approach, RSE encompasses terrestrial, oceanic, and atmospheric sensing.
The journal emphasizes biophysical and quantitative approaches to remote sensing at local to global scales, covering a diverse range of applications and techniques.
RSE serves as a vital platform for the exchange of knowledge and advancements in the dynamic field of remote sensing.