MuSRFM: Multiple scale resolution fusion based precise and robust satellite derived bathymetry model for island nearshore shallow water regions using sentinel-2 multi-spectral imagery
Xiaoming Qin , Ziyin Wu , Xiaowen Luo , Jihong Shang , Dineng Zhao , Jieqiong Zhou , Jiaxin Cui , Hongyang Wan , Guochang Xu
{"title":"MuSRFM: Multiple scale resolution fusion based precise and robust satellite derived bathymetry model for island nearshore shallow water regions using sentinel-2 multi-spectral imagery","authors":"Xiaoming Qin , Ziyin Wu , Xiaowen Luo , Jihong Shang , Dineng Zhao , Jieqiong Zhou , Jiaxin Cui , Hongyang Wan , Guochang Xu","doi":"10.1016/j.isprsjprs.2024.09.007","DOIUrl":null,"url":null,"abstract":"<div><p>The multi-spectral imagery based Satellite Derived Bathymetry (SDB) provides an efficient and cost-effective approach for acquiring bathymetry data of nearshore shallow water regions. Compared with conventional pixelwise inversion models, Deep Learning (DL) models have the theoretical capability to encompass a broader receptive field, automatically extracting comprehensive spatial features. However, enhancing spatial features by increasing the input size escalates computational complexity and model scale, challenging the hardware. To address this issue, we propose the Multiple Scale Resolution Fusion Model (MuSRFM), a novel DL-based SDB model, to integrate information of varying scales by utilizing temporally fused Sentinel-2 L2A multi-spectral imagery. The MuSRFM uses a Multi-scale Center-aligned Hierarchical Resampler (MCHR) to composite large-scale multi-spectral imagery into hierarchical scale resolution representations since the receptive field gradually narrows its focus as the spatial resolution decreases. Through this strategy, the MuSRFM gains access to rich spatial information while maintaining efficiency by progressively aggregating features of different scales through the Cropped Aligned Fusion Module (CAFM). We select St. Croix (Virgin Islands) as the training/testing dataset source, and the Root Mean Square Error (RMSE) obtained by the MuSRFM on the testing dataset is 0.8131 m (with a bathymetric range of 0–25 m), surpassing the machine learning based models and traditional semi-empirical models used as the baselines by over 35 % and 60 %, respectively. Additionally, multiple island areas worldwide, including Vieques, Oahu, Kauai, Saipan and Tinian, which exhibit distinct characteristics, are utilized to construct a real-world dataset for assessing the generalizability and transferability of the proposed MuSRFM. While the MuSRFM experiences a degradation in accuracy when applied to the diverse real-world dataset, it outperforms other baseline models considerably. Across various study areas in the real-world dataset, its RMSE lead over the second-ranked model ranges from 6.8 % to 38.1 %, indicating its accuracy and generalizability; in the Kauai area, where the performance is not ideal, a significant improvement in accuracy is achieved through fine-tuning on limited in-situ data. The code of MuSRFM is available at <span><span>https://github.com/qxm1995716/musrfm</span><svg><path></path></svg></span>.</p></div>","PeriodicalId":50269,"journal":{"name":"ISPRS Journal of Photogrammetry and Remote Sensing","volume":"218 ","pages":"Pages 150-169"},"PeriodicalIF":10.6000,"publicationDate":"2024-09-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S0924271624003459/pdfft?md5=4925ae29c5fd595f63a6ca31611a8d4c&pid=1-s2.0-S0924271624003459-main.pdf","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"ISPRS Journal of Photogrammetry and Remote Sensing","FirstCategoryId":"5","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0924271624003459","RegionNum":1,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"GEOGRAPHY, PHYSICAL","Score":null,"Total":0}
引用次数: 0
Abstract
The multi-spectral imagery based Satellite Derived Bathymetry (SDB) provides an efficient and cost-effective approach for acquiring bathymetry data of nearshore shallow water regions. Compared with conventional pixelwise inversion models, Deep Learning (DL) models have the theoretical capability to encompass a broader receptive field, automatically extracting comprehensive spatial features. However, enhancing spatial features by increasing the input size escalates computational complexity and model scale, challenging the hardware. To address this issue, we propose the Multiple Scale Resolution Fusion Model (MuSRFM), a novel DL-based SDB model, to integrate information of varying scales by utilizing temporally fused Sentinel-2 L2A multi-spectral imagery. The MuSRFM uses a Multi-scale Center-aligned Hierarchical Resampler (MCHR) to composite large-scale multi-spectral imagery into hierarchical scale resolution representations since the receptive field gradually narrows its focus as the spatial resolution decreases. Through this strategy, the MuSRFM gains access to rich spatial information while maintaining efficiency by progressively aggregating features of different scales through the Cropped Aligned Fusion Module (CAFM). We select St. Croix (Virgin Islands) as the training/testing dataset source, and the Root Mean Square Error (RMSE) obtained by the MuSRFM on the testing dataset is 0.8131 m (with a bathymetric range of 0–25 m), surpassing the machine learning based models and traditional semi-empirical models used as the baselines by over 35 % and 60 %, respectively. Additionally, multiple island areas worldwide, including Vieques, Oahu, Kauai, Saipan and Tinian, which exhibit distinct characteristics, are utilized to construct a real-world dataset for assessing the generalizability and transferability of the proposed MuSRFM. While the MuSRFM experiences a degradation in accuracy when applied to the diverse real-world dataset, it outperforms other baseline models considerably. Across various study areas in the real-world dataset, its RMSE lead over the second-ranked model ranges from 6.8 % to 38.1 %, indicating its accuracy and generalizability; in the Kauai area, where the performance is not ideal, a significant improvement in accuracy is achieved through fine-tuning on limited in-situ data. The code of MuSRFM is available at https://github.com/qxm1995716/musrfm.
期刊介绍:
The ISPRS Journal of Photogrammetry and Remote Sensing (P&RS) serves as the official journal of the International Society for Photogrammetry and Remote Sensing (ISPRS). It acts as a platform for scientists and professionals worldwide who are involved in various disciplines that utilize photogrammetry, remote sensing, spatial information systems, computer vision, and related fields. The journal aims to facilitate communication and dissemination of advancements in these disciplines, while also acting as a comprehensive source of reference and archive.
P&RS endeavors to publish high-quality, peer-reviewed research papers that are preferably original and have not been published before. These papers can cover scientific/research, technological development, or application/practical aspects. Additionally, the journal welcomes papers that are based on presentations from ISPRS meetings, as long as they are considered significant contributions to the aforementioned fields.
In particular, P&RS encourages the submission of papers that are of broad scientific interest, showcase innovative applications (especially in emerging fields), have an interdisciplinary focus, discuss topics that have received limited attention in P&RS or related journals, or explore new directions in scientific or professional realms. It is preferred that theoretical papers include practical applications, while papers focusing on systems and applications should include a theoretical background.