Qunming Wang , Jiayi Li , Xiaohua Tong , Peter M. Atkinson
{"title":"TSI-Siamnet: A Siamese network for cloud and shadow detection based on time-series cloudy images","authors":"Qunming Wang , Jiayi Li , Xiaohua Tong , Peter M. Atkinson","doi":"10.1016/j.isprsjprs.2024.05.022","DOIUrl":null,"url":null,"abstract":"<div><p>Accurate cloud and shadow detection is a crucial prerequisite for optical remote sensing image analysis and application. Multi-temporal-based cloud and shadow detection methods are a preferable choice to detect clouds in complex scenes (e.g., thin clouds, broken clouds and clouds with interference from artificial surfaces with high reflectivity). However, such methods commonly require cloud-free reference images, and this may be difficult to achieve in time-series data since clouds are often prevalent and of varying spatial distribution in optical remote sensing images. Furthermore, current multi-temporal-based methods have limited feature extraction capability and rely heavily on prior assumptions. To address these issues, this paper proposes a Siamese network (Siamnet) for cloud and shadow detection based on Time-Series cloudy Images, namely TSI-Siamnet, which consists of two steps: 1) low-rank and sparse component decomposition of time-series cloudy images is conducted to construct a composite reference image to cope with dynamic changes in the cloud distribution in time-series images; 2) an extended Siamnet with optimal difference calculation module (DM) and multi-scale difference features fusion module (MDFM) is constructed to extract reliable disparity features and alleviate semantic information feature dilution during the decoder part. TSI-Siamnet was tested extensively on seven land cover types in the well-known Landsat 8 Biome dataset. Compared to six state-of-the-art methods (including four deep learning-based methods and two classical non-deep learning-based methods), TSI-Siamnet produced the best performance with an overall accuracy of 95.05% and MIoU of 84.37%. In three more challenging experiments, TSI-Siamnet showed enhanced detection of thin and broken clouds and greater anti-interference to highly reflective surfaces. TSI-Siamnet provides a novel strategy to explore comprehensively the valid information in time-series cloudy images and integrate the extracted spectral-spatial–temporal features for reliable cloud and shadow detection.</p></div>","PeriodicalId":50269,"journal":{"name":"ISPRS Journal of Photogrammetry and Remote Sensing","volume":null,"pages":null},"PeriodicalIF":10.6000,"publicationDate":"2024-06-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"ISPRS Journal of Photogrammetry and Remote Sensing","FirstCategoryId":"5","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0924271624002260","RegionNum":1,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"GEOGRAPHY, PHYSICAL","Score":null,"Total":0}
引用次数: 0
Abstract
Accurate cloud and shadow detection is a crucial prerequisite for optical remote sensing image analysis and application. Multi-temporal-based cloud and shadow detection methods are a preferable choice to detect clouds in complex scenes (e.g., thin clouds, broken clouds and clouds with interference from artificial surfaces with high reflectivity). However, such methods commonly require cloud-free reference images, and this may be difficult to achieve in time-series data since clouds are often prevalent and of varying spatial distribution in optical remote sensing images. Furthermore, current multi-temporal-based methods have limited feature extraction capability and rely heavily on prior assumptions. To address these issues, this paper proposes a Siamese network (Siamnet) for cloud and shadow detection based on Time-Series cloudy Images, namely TSI-Siamnet, which consists of two steps: 1) low-rank and sparse component decomposition of time-series cloudy images is conducted to construct a composite reference image to cope with dynamic changes in the cloud distribution in time-series images; 2) an extended Siamnet with optimal difference calculation module (DM) and multi-scale difference features fusion module (MDFM) is constructed to extract reliable disparity features and alleviate semantic information feature dilution during the decoder part. TSI-Siamnet was tested extensively on seven land cover types in the well-known Landsat 8 Biome dataset. Compared to six state-of-the-art methods (including four deep learning-based methods and two classical non-deep learning-based methods), TSI-Siamnet produced the best performance with an overall accuracy of 95.05% and MIoU of 84.37%. In three more challenging experiments, TSI-Siamnet showed enhanced detection of thin and broken clouds and greater anti-interference to highly reflective surfaces. TSI-Siamnet provides a novel strategy to explore comprehensively the valid information in time-series cloudy images and integrate the extracted spectral-spatial–temporal features for reliable cloud and shadow detection.
期刊介绍:
The ISPRS Journal of Photogrammetry and Remote Sensing (P&RS) serves as the official journal of the International Society for Photogrammetry and Remote Sensing (ISPRS). It acts as a platform for scientists and professionals worldwide who are involved in various disciplines that utilize photogrammetry, remote sensing, spatial information systems, computer vision, and related fields. The journal aims to facilitate communication and dissemination of advancements in these disciplines, while also acting as a comprehensive source of reference and archive.
P&RS endeavors to publish high-quality, peer-reviewed research papers that are preferably original and have not been published before. These papers can cover scientific/research, technological development, or application/practical aspects. Additionally, the journal welcomes papers that are based on presentations from ISPRS meetings, as long as they are considered significant contributions to the aforementioned fields.
In particular, P&RS encourages the submission of papers that are of broad scientific interest, showcase innovative applications (especially in emerging fields), have an interdisciplinary focus, discuss topics that have received limited attention in P&RS or related journals, or explore new directions in scientific or professional realms. It is preferred that theoretical papers include practical applications, while papers focusing on systems and applications should include a theoretical background.