{"title":"Deeper and Broader Multimodal Fusion: Cascaded Forest-of-Experts for Land Cover Classification","authors":"Guangxia Wang;Kuiliang Gao;Xiong You","doi":"10.1109/LGRS.2024.3516854","DOIUrl":"https://doi.org/10.1109/LGRS.2024.3516854","url":null,"abstract":"Multimodal land cover classification (LCC) of optical and SAR images has become a research hotspot. However, there are still two unsolved problems: the lack of a deep fusion mechanism and the neglect of the diversity of multimodal features. Inspired by ensemble learning, this letter proposes the cascaded multimodal forest-of-experts (CM2FEs) for deeper and broader fusion to further improve the performance of LCC. The proposed method first establishes the expert tree, then combines multiple trees at the same level into a forest, and finally forms a cascaded forest across different levels. Specifically, the novel designs include three points: 1) the multimodal expert tree is built based on linear projection and dynamic routing, with multiple layers of experts; it can acquire more discriminative multimodal features through deeper fusion; 2) the cascaded forest is formed by combining expert trees at the same level and different levels, which can effectively ensemble the knowledge learned by different trees; it can generate more diverse multimodal features through broader fusion; and 3) two expert exchange strategies are proposed to transfer knowledge between different trees and further optimize the feature fusion effect. Experiments show that the proposed method performs better than existing methods, and the mean IoU (mIoU) has been improved by at least 1.60%–3.25%.","PeriodicalId":91017,"journal":{"name":"IEEE geoscience and remote sensing letters : a publication of the IEEE Geoscience and Remote Sensing Society","volume":"22 ","pages":"1-5"},"PeriodicalIF":0.0,"publicationDate":"2024-12-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142858948","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Impact of Targeted Sounding Observations From FY-4B GIIRS on Two Super Typhoon Forecasts in 2024","authors":"Ruoying Yin;Wei Han","doi":"10.1109/LGRS.2024.3516004","DOIUrl":"https://doi.org/10.1109/LGRS.2024.3516004","url":null,"abstract":"Hyperspectral infrared (IR) sounders on board geostationary (GEO) satellite, represented by the GEO interferometric IR sounder (GIIRS), can provide unprecedented and valuable observations with high-temporal resolution for disaster prevention, mitigation, and meteorological support services. FengYun-4B (FY-4B) GIIRS has carried out targeted sounding observations with a temporal resolution of 15 min for two super typhoons in 2024 (Typhoon Gaemi and Typhoon Yagi). In this study, an operational parallel experiment system was established to assimilate the FY-4B GIIRS targeted radiances in real time, and the diagnosis and analysis of typhoon forecast were carried out after the typhoon dissipated. The results indicate that assimilating FY-4B GIIRS can improve the track forecast of the two super typhoons in real-time operational environment, and the improvement is more significant after 60-h forecast. The average track forecast was increased by 22.5% in Typhoon Gaemi and by 6.3% in Typhoon Yagi, although the impact on typhoon intensity forecast was not significant. Additionally, the difference in the number of clear sky points assimilated in the two case experiments shows the importance of cloud sky assimilation in the future. This study reveals the potential and value of FY-4B GIIRS quantitative assimilation application to improve numerical weather prediction (NWP) skills, especially high-impact weather (HIW) events.","PeriodicalId":91017,"journal":{"name":"IEEE geoscience and remote sensing letters : a publication of the IEEE Geoscience and Remote Sensing Society","volume":"22 ","pages":"1-5"},"PeriodicalIF":0.0,"publicationDate":"2024-12-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10794787","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142858940","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jiajun Yang;Wenjing Wang;Keyan Chen;Liqin Liu;Zhengxia Zou;Zhenwei Shi
{"title":"Structural Representation-Guided GAN for Remote Sensing Image Cloud Removal","authors":"Jiajun Yang;Wenjing Wang;Keyan Chen;Liqin Liu;Zhengxia Zou;Zhenwei Shi","doi":"10.1109/LGRS.2024.3516078","DOIUrl":"https://doi.org/10.1109/LGRS.2024.3516078","url":null,"abstract":"Optical remote sensing imagery is often compromised by cloud cover, making effective cloud-removal techniques essential for enhancing the usability of such data. We designed a novel structural representation-guided generative adversarial network (GAN) framework for cloud removal, in which structure and gradient branches are integrated into the network, helping the model focus on the structural representations of ground objects during image reconstruction. Different from previous methods that concentrate on recovering pixel information, we emphasize learning the structural information of remote sensing images. We then utilize error feedback to fuse features from the structural auxiliary branch, guiding the image reconstruction process. During the training phase, synthetic cloud images are used to supervise the optimization of the cloud-removal network, while real cloud images are employed in an adversarial training manner for unsupervised learning to improve the generalization ability of the network. Additionally, multitemporal revisit images from remote sensing satellites are employed as auxiliary inputs, aiding the network to remove thick clouds reliably. We evaluated our framework on a dataset derived from SEN12MS-CR, and the proposed method outperformed classical cloud-removal methods in both objective performance and subjective visual quality. Furthermore, compared to other methods, our approach achieved superior cloud-removal results on real images.","PeriodicalId":91017,"journal":{"name":"IEEE geoscience and remote sensing letters : a publication of the IEEE Geoscience and Remote Sensing Society","volume":"22 ","pages":"1-5"},"PeriodicalIF":0.0,"publicationDate":"2024-12-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142858946","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Satellite Selection Algorithm for GNSS-R InSAR Elevation Deformation Retrieval","authors":"Jingyan Yu;Yunlong Zhu;Zhixin Deng;Yanling Zhao","doi":"10.1109/LGRS.2024.3514913","DOIUrl":"https://doi.org/10.1109/LGRS.2024.3514913","url":null,"abstract":"The global navigation satellite system (GNSS) reflectometry synthetic aperture radar (SAR) interferometry (GNSS-R InSAR) system enables elevation deformation retrieval using a single satellite. However, variations in bistatic configurations and the generally low accuracy of most satellites necessitate a refined satellite selection method. Thus, this letter proposes a satellite selection algorithm for GNSS-R InSAR, aiming to optimize satellite selection and data acquisition time to improve the precision of elevation deformation monitoring. First, the interferometric phase model based on the repeat-pass concept was established using GPS L5 signals. Second, a satellite selection algorithm was proposed that incorporates constraints on resolution cells, spatial baseline, and phase sensitivity for elevation deformation, derived from an analysis of the repeat-pass spatial baseline of GNSS satellites, interferometric phase sensitivity, and the maximum deformation range. Third, 24 sets of repeat-pass data were collected, and the experimental results validate the effectiveness of this single-satellite selection approach.","PeriodicalId":91017,"journal":{"name":"IEEE geoscience and remote sensing letters : a publication of the IEEE Geoscience and Remote Sensing Society","volume":"22 ","pages":"1-5"},"PeriodicalIF":0.0,"publicationDate":"2024-12-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142844386","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Fast Fusion Method for Multi- and Hyperspectral Images via Subpixel-Shift Decomposition","authors":"Jingwei Deng;Xiaolin Han;Huan Zhang;Weidong Sun","doi":"10.1109/LGRS.2024.3515207","DOIUrl":"https://doi.org/10.1109/LGRS.2024.3515207","url":null,"abstract":"Several spectral and spatial dictionary-based methods exist for fusing a high-spatial-resolution multispectral image (HR-MSI) with a low-spatial-resolution hyperspectral image (LR-HSI). However, using only one type of dictionary is insufficient to preserve spatial and spectral information simultaneously, while utilizing both dictionaries would increase the computational costs. To address this problem, we propose a fast fusion method (called FFD) for HR-MSIs and LR-HSIs via subpixel-shift decomposition. In this method, through joint optimization of low rank and sparsity within the framework of subpixel shift and sparse representation, an ultimate spectral dictionary is acquired along with its associated coefficients. Specifically, the HR-MSI is decomposed into subimage sequences of the same spatial resolution as the LR-HSI first, to replace the use of spatial dictionaries. Subsequently, a new fusion model is constructed based on this decomposition incorporating the constraints of low rank and sparsity, and especially, a low-rank term is introduced to constrain the spectral consistency along the decomposition direction. Then, the model is theoretically derived by using the alternating direction method of multipliers (ADMM) method, and a joint optimization for the spectral dictionary and its sparse coefficients is obtained. Finally, the desired HR-HSI can be reconstructed by using the above fused subimages through a simple inversed composition. Experimental results on different datasets show that compared with the other related methods, our proposed FFD can achieve an equivalent fusion effect to the best of them in a much shorter time.","PeriodicalId":91017,"journal":{"name":"IEEE geoscience and remote sensing letters : a publication of the IEEE Geoscience and Remote Sensing Society","volume":"22 ","pages":"1-5"},"PeriodicalIF":0.0,"publicationDate":"2024-12-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142858939","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Jingliang Guo;Mengke Yuan;Tong Wang;Zhifeng Li;Xiaohong Jia;Dong-Ming Yan
{"title":"DTESR: Remote Sensing Imagery Super-Resolution With Dynamic Reference Textures Exploitation","authors":"Jingliang Guo;Mengke Yuan;Tong Wang;Zhifeng Li;Xiaohong Jia;Dong-Ming Yan","doi":"10.1109/LGRS.2024.3515136","DOIUrl":"https://doi.org/10.1109/LGRS.2024.3515136","url":null,"abstract":"Reference-based remote sensing super-resolution (RefRS-SR) method shows great potential for improving both spatial resolution and coverage area of remote sensing images, by which high-resolution (HR) reference images can supplement fine details for low-resolution (LR) but wide coverage images. However, most RefRS-SR methods treat the reference as a static template and unidirectionally transfer the high-frequency information to the LR input. To address the issue of inefficient and inaccurate guided super-resolving, we propose a new RefRS-SR method with dynamic reference textures exploitation dubbed DTESR. The key referenced restoration (Ref Restoration) module consists of three components: correlation generation, texture enhancement and refinement (TER), and adaptive similarity-based fusion to progressively reconstruct high correlation and delicate textures for the LR input. Specifically, both the LR input and reference features are utilized for precise correlation generation. Next, both features are enhanced and refined with the most suitable reference under the guidance of the correlation map. Moreover, a learnable fusion method is designed to maintain the consistency of adjacent pixels. These operations will be iteratively applied to the three reconstruction scales to promote the exploitation of the Ref features. Through comprehensive quantitative and qualitative evaluations, our experimental results demonstrate that DTESR surpasses the current state-of-the-art RefRS-SR methods.","PeriodicalId":91017,"journal":{"name":"IEEE geoscience and remote sensing letters : a publication of the IEEE Geoscience and Remote Sensing Society","volume":"22 ","pages":"1-5"},"PeriodicalIF":0.0,"publicationDate":"2024-12-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142858947","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"A Self-Supervised Pretraining Framework for Context-Aware Building Edge Extraction From 3-D Point Clouds","authors":"Hongxin Yang;Shanshan Xu;Sheng Xu","doi":"10.1109/LGRS.2024.3514857","DOIUrl":"https://doi.org/10.1109/LGRS.2024.3514857","url":null,"abstract":"Building edge points, as essential geometric features, are crucial for advancing smart city initiatives and ensuring the precise reconstruction of 3-D structures. However, existing methods struggle to effectively design point-to-edge distance constraints for accurate building edge point identification. In this letter, we propose a novel self-supervised learning (SSL)-based pretraining framework that integrates an innovative edge point identification loss function for extracting building edge points. Specifically, we use an SSL-based feature extractor, leveraging a masked autoencoder to generate pointwise features from the input building point clouds. These features are subsequently processed by the proposed edge point identification module, which optimizes three key distance-based loss functions: the distance between any input point and its nearest edge, the distance between candidate edge points and the projection of the input point, and the distance between candidate edge points and the edges themselves. The proposed framework demonstrates superior performance in edge point extraction across both partial and complete datasets, outperforming existing methods in edge point identification.","PeriodicalId":91017,"journal":{"name":"IEEE geoscience and remote sensing letters : a publication of the IEEE Geoscience and Remote Sensing Society","volume":"22 ","pages":"1-5"},"PeriodicalIF":0.0,"publicationDate":"2024-12-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142858949","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ruohan Li;Dongdong Wang;Sadashiva Devadiga;Sudipta Sarkar;Miguel O. Román
{"title":"MCD18 V6.2: A New Version of MODIS Downward Shortwave Radiation and Photosynthetically Active Radiation Products","authors":"Ruohan Li;Dongdong Wang;Sadashiva Devadiga;Sudipta Sarkar;Miguel O. Román","doi":"10.1109/LGRS.2024.3507822","DOIUrl":"https://doi.org/10.1109/LGRS.2024.3507822","url":null,"abstract":"This study presents the new version of MODIS/Terra + Aqua Surface Radiation Daily/3-h downward shortwave radiation (DSR) (MCD18A1 V6.2) and photosynthetic active radiation (PAR) (MCD18A2 V6.2) product generated by MODIS adaptive processing system (MODAPS) using the latest version of the science algorithm developed by the NASA MODIS land science team. Key improvements in the new algorithm include using multiple bands covering visible, near-infrared, and shortwave infrared to enhance the capability of characterizing cloud optical characteristics, especially over snow-covered surfaces, and adopting linear interpolation for temporal scaling from instantaneous to 3-hourly retrievals. Comparative validation against MCD18 V6.1 and clouds and the Earth’s radiant energy system synoptic (CERES-SYN) demonstrates that V6.2 significantly improves accuracy at instantaneous, 3-hourly, and daily scales, particularly in snow-covered regions. The root mean square error (RMSE) (relative RMSE: rRMSE) of V6.2 reaches 101.9 W/m2 (18.8%) and 48.4 W/m2 (20.8%) for instantaneous DSR and PAR. The RMSE (rRMSE) reaches 29.9 W/m2 (16.9%) and 14.1 W/m2 (18.4%) for daily DSR and PAR, respectively. Aggregated to 100 km, V6.2 matches CERES-SYN accuracy using only polar-orbiting satellite data. This study also explores the potential for future improvement by integrating geostationary observations to enhance accuracy further.","PeriodicalId":91017,"journal":{"name":"IEEE geoscience and remote sensing letters : a publication of the IEEE Geoscience and Remote Sensing Society","volume":"22 ","pages":"1-5"},"PeriodicalIF":0.0,"publicationDate":"2024-12-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142821190","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Efficient Phase Congruency-Based Feature Transform for Rapid Matching of Planetary Remote Sensing Images","authors":"Genyi Wan;Rong Huang;Yusheng Xu;Zhen Ye;Qionghua You;Xiongfeng Yan;Xiaohua Tong","doi":"10.1109/LGRS.2024.3510794","DOIUrl":"https://doi.org/10.1109/LGRS.2024.3510794","url":null,"abstract":"Plenty of effort has been devoted to solving the nonlinear radiation distortions (NRDs) in planetary image matching. The mainstream solutions convert multimodal images into “single” modal images, which requires building the intermediate modalities of images. Phase congruency (PC) features have been widely used to construct intermediate modalities due to their excellent structure extraction capabilities and have proven their effectiveness on Earth remote sensing images. However, when dealing with large-scale planetary remote sensing images (PRSIs), traditional PC features constructed based on the log-Gabor filter take considerable time, counterproductive to global topographic mapping. To address the efficiency issue, this work proposes a fast planetary image-matching method based on efficient PC-based feature transform (EPCFT). Specifically, we introduce a method to calculate PC using Gaussian first- and second-order derivatives, called efficient PC (EPC). Different from the log-Gabor filter, which is sensitive to structures in a single direction, \u0000<inline-formula> <tex-math>$rm EPC$ </tex-math></inline-formula>\u0000 uses circularly symmetric filters to equally process changes in all directions. The experiments with 100 image pairs show that compared with other methods, the efficiency of our method is nearly doubled without loss of accuracy.","PeriodicalId":91017,"journal":{"name":"IEEE geoscience and remote sensing letters : a publication of the IEEE Geoscience and Remote Sensing Society","volume":"22 ","pages":"1-5"},"PeriodicalIF":0.0,"publicationDate":"2024-12-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142858941","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Gaussian-Inspired Attention Mechanism for Hyperspectral Anomaly Detection","authors":"Ruike Wang;Jing Hu","doi":"10.1109/LGRS.2024.3514166","DOIUrl":"https://doi.org/10.1109/LGRS.2024.3514166","url":null,"abstract":"Hyperspectral anomaly detection (HAD) aims to identify spectrally distinct pixels within a hyperspectral image (HSI). This task necessitates capturing both local spectral information and spatial smoothness, posing a significant challenge for traditional methods. This letter proposes a novel autoencoder framework that leverages a Gaussian-inspired attention mechanism to address this challenge effectively. Specifically, we introduce a novel Gaussian attention layer embedded within the encoder. This layer utilizes a learnable Gaussian kernel to prioritize the local neighborhood of each pixel. This approach effectively captures fine-grained features crucial for background reconstruction. The learned representations are then passed through a deep autoencoder architecture to reconstruct anomaly-free data. Pixels with significant reconstruction errors are subsequently flagged as anomalies. Experiments on several datasets demonstrate the effectiveness of the proposed approach. Compared to existing methods, our framework achieves superior performance in terms of detection accuracy. This finding highlights the potential of Gaussian-inspired attention mechanisms for enhancing HAD. The code is released at: \u0000<uri>https://github.com/rk-rkk/Gaussian-Inspired-Attention-Mechanism-for-Hyperspectral-Anomaly-Detection</uri>\u0000.","PeriodicalId":91017,"journal":{"name":"IEEE geoscience and remote sensing letters : a publication of the IEEE Geoscience and Remote Sensing Society","volume":"22 ","pages":"1-5"},"PeriodicalIF":0.0,"publicationDate":"2024-12-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142844381","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}