ISPRS Journal of Photogrammetry and Remote Sensing最新文献

{"title":"Homogeneous tokenizer matters: Homogeneous visual tokenizer for remote sensing image understanding","authors":"Run Shao ,&nbsp;Zhaoyang Zhang ,&nbsp;Chao Tao ,&nbsp;Yunsheng Zhang ,&nbsp;Chengli Peng ,&nbsp;Haifeng Li","doi":"10.1016/j.isprsjprs.2024.09.009","DOIUrl":"10.1016/j.isprsjprs.2024.09.009","url":null,"abstract":"<div><p>On the basis of the transformer architecture and the pretext task of “next-token prediction”, multimodal large language models (MLLMs) are revolutionizing the paradigm in the field of remote sensing image understanding. However, the tokenizer, as one of the fundamental components of MLLMs, has long been overlooked or even misunderstood in visual tasks. A key factor contributing to the great comprehension power of large language models is that natural language tokenizers utilize meaningful words or subwords as the basic elements of language. In contrast, mainstream visual tokenizers, represented by patch-based methods such as Patch Embed, rely on meaningless rectangular patches as basic elements of vision. Analogous to words or subwords in language, we define semantically independent regions (SIRs) for vision and then propose two properties that an ideal visual tokenizer should possess: (1) homogeneity, where SIRs serve as the basic elements of vision, and (2) adaptivity, which allows for a flexible number of tokens to accommodate images of any size and tasks of any granularity. On this basis, we design a simple HOmogeneous visual tOKenizer: HOOK. HOOK consists of two modules: an object perception module (OPM) and an object vectorization module (OVM). To achieve homogeneity, the OPM splits the image into 4 × 4 pixel seeds and then uses a self-attention mechanism to identify SIRs. The OVM employs cross-attention to merge seeds within the same SIR. To achieve adaptability, the OVM predefines a variable number of learnable vectors as cross-attention queries, allowing for the adjustment of the token quantity. We conducted experiments on the NWPU-RESISC45, WHU-RS19, and NaSC-TG2 classification datasets for sparse tasks and the GID5 and DGLCC segmentation datasets for dense tasks. The results show that the visual tokens obtained by HOOK correspond to individual objects, thereby verifying their homogeneity. Compared with randomly initialized or pretrained Patch Embed, which required more than one hundred tokens per image, HOOK required only 6 and 8 tokens for sparse and dense tasks, respectively, resulting in performance improvements of 2% to 10% and efficiency improvements of 1.5 to 2.8 times. The homogeneity and adaptability of the proposed approach provide new perspectives for the study of visual tokenizers. Guided by these principles, the developed HOOK has the potential to replace traditional Patch Embed. The code is available at <span><span>https://github.com/GeoX-Lab/Hook</span><svg><path></path></svg></span>.</p></div>","PeriodicalId":50269,"journal":{"name":"ISPRS Journal of Photogrammetry and Remote Sensing","volume":"218 ","pages":"Pages 294-310"},"PeriodicalIF":10.6,"publicationDate":"2024-09-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142270436","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"VDFT: Robust feature matching of aerial and ground images using viewpoint-invariant deformable feature transformation","authors":"Bai Zhu ,&nbsp;Yuanxin Ye ,&nbsp;Jinkun Dai ,&nbsp;Tao Peng ,&nbsp;Jiwei Deng ,&nbsp;Qing Zhu","doi":"10.1016/j.isprsjprs.2024.09.016","DOIUrl":"10.1016/j.isprsjprs.2024.09.016","url":null,"abstract":"<div><p>Establishing accurate correspondences between aerial and ground images is facing immense challenges because of the drastic viewpoint, illumination, and scale variations resulting from significant differences in viewing angles, shoot timing, and imaging mechanisms. To cope with these issues, we propose an effective aerial-to-ground feature matching method, named Viewpoint-invariant Deformable Feature Transformation (VDFT), which aims to comprehensively enhance the discrimination of local features by utilizing deformable convolutional network (DCN) and seed attention mechanism. Specifically, the proposed VDFT is constructed consisting of three pivotal modules: (1) a learnable deformable feature network is established by using DCN and Depthwise Separable Convolution (DSC) to obtain dynamic receptive fields, addressing local geometric deformations caused by viewpoint variation; (2) an improved joint detection and description strategy is presented through concurrently sharing the multi-level deformable feature representation to enhance the localization accuracy and representation capabilities of feature points; and (3) a seed attention matching module is built by introducing self- and cross- seed attention mechanisms to improve the performance and efficiency for aerial-to-ground feature matching. Finally, we conduct thorough experiments to verify the matching performance of our VDFT on five challenging aerial-to-ground datasets. Extensive experimental evaluations prove that our VDFT is more resistant to perspective distortion and drastic variations in viewpoint, illumination, and scale. It exhibits satisfactory matching performance and outperforms the current state-of-the-art (SOTA) methods in terms of robustness and accuracy.</p></div>","PeriodicalId":50269,"journal":{"name":"ISPRS Journal of Photogrammetry and Remote Sensing","volume":"218 ","pages":"Pages 311-325"},"PeriodicalIF":10.6,"publicationDate":"2024-09-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142270302","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A cost-effective and robust mapping method for diverse crop types using weakly supervised semantic segmentation with sparse point samples","authors":"Zhiwen Cai ,&nbsp;Baodong Xu ,&nbsp;Qiangyi Yu ,&nbsp;Xinyu Zhang ,&nbsp;Jingya Yang ,&nbsp;Haodong Wei ,&nbsp;Shiqi Li ,&nbsp;Qian Song ,&nbsp;Hang Xiong ,&nbsp;Hao Wu ,&nbsp;Wenbin Wu ,&nbsp;Zhihua Shi ,&nbsp;Qiong Hu","doi":"10.1016/j.isprsjprs.2024.09.017","DOIUrl":"10.1016/j.isprsjprs.2024.09.017","url":null,"abstract":"<div><p>Accurate and timely information on the spatial distribution and areas of crop types is critical for yield estimation, agricultural management, and sustainable development. However, traditional crop classification methods often struggle to identify various crop types effectively due to their intricate spatiotemporal patterns and high training data demands. To address this challenge, we developed a <strong>Struct</strong>ure-aware <strong>Lab</strong>el e<strong>X</strong>pansion segmentation Network (StructLabX-Net) for diverse crop type mapping using limited point-annotated samples. StructLabX-Net features a backbone U-TempoNet, which combines CNNs and LSTM to explore intricate spatiotemporal patterns. It also incorporates multi-task weak supervision heads for edge detection and pseudo-label expansion, adding crucial structure and contextual insights. We tested the StructLabX-Net across three distinct regions in China, assessing over 10 crop types and comparing its performance against five popular classifiers based on multi-temporal Sentinel-2 images. The results showed that StructLabX-Net significantly outperformed RF, SVM, DeepCropMapping, Transformer, and patch-based CNN in identifying various crop types across three regions with sparse training samples. It achieved the highest overall accuracy and mean <em>F1-score</em>: 91.0% and 89.1% in Jianghan Plain, 91.5% and 90.7% in Songnen Plain, as well as 91.0% and 90.8% in Sanjiang Plain. StructLabX-Net demonstrated a particular advantage for those “hard types” characterized by limited samples and complex phenological features. Furthermore, ablation experiments highlight the crucial role of the “edge” head in guiding the model to accurately differentiate between various crop types with clearer class boundaries, and the “expansion” head in refining the understanding of target crops by providing extra details in pseudo-labels. Meanwhile, combining our backbone U-TempoNet with multi-task weak supervision heads exhibited superior results of crop type mapping than those derived by other segmentation models. Overall, StructLabX-Net maximizes the utilization of limited sparse samples from field surveys, offering a simple, cost-effective, and robust solution for accurately mapping various crop types at large scales. The code will be publicly available at <span><span>https://github.com/BruceKai/StructLabX-Net</span><svg><path></path></svg></span>.</p></div>","PeriodicalId":50269,"journal":{"name":"ISPRS Journal of Photogrammetry and Remote Sensing","volume":"218 ","pages":"Pages 260-276"},"PeriodicalIF":10.6,"publicationDate":"2024-09-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142270434","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"SoftFormer: SAR-optical fusion transformer for urban land use and land cover classification","authors":"Rui Liu ,&nbsp;Jing Ling ,&nbsp;Hongsheng Zhang","doi":"10.1016/j.isprsjprs.2024.09.012","DOIUrl":"10.1016/j.isprsjprs.2024.09.012","url":null,"abstract":"<div><p>Classification of urban land use and land cover is vital to many applications, and naturally becomes a popular topic in remote sensing. The finite information carried by unimodal data, the compound land use types, and the poor signal-noise ratio caused by restricted weather conditions would inevitably lead to relatively poor classification performance. Recently in remote sensing society, multimodal data fusion with deep learning technology has gained a great deal of attention. Existing research exhibit integration of multimodal data at a single level, while simultaneously lacking exploration of the immense potential provided by popular transformer and CNN structures for effectively leveraging multimodal data, which may fall into the trap that makes the information fusion inadequate. We introduce SoftFormer, a novel network that synergistically merges the strengths of CNNs with transformers, as well as achieving multi-level fusion. To extract local features from images, we propose an innovative mechanism called Interior Self-Attention, which is seamlessly integrated into the backbone network. To fully exploit the global semantic information from both modalities, in the feature-level fusion, we introduce a joint key–value learning fusion approach to integrate multimodal data within a unified semantic space. The decision and feature level information are simultaneously integrated, resulting in a multi-level fusion transformer network. Results on four remote sensing datasets show that SoftFormer is able to achieve at least 1.32%, 0.7%, and 0.99% performance improvement in overall accuracy, kappa index, and mIoU, compared to other state-of-the-art methods, the ablation studies show that multimodal fusion outperforms the unimodal data on urban land cover and land use classification, the highest overall accuracy, kappa index as well as mIoU improvement can be up to 5.71%, 10.32% and 7.91%, and the proposed modules are able to boost performance to some extent, even with cloud cover. Code will be publicly available at <span><span>https://github.com/rl1024/SoftFormer</span><svg><path></path></svg></span>.</p></div>","PeriodicalId":50269,"journal":{"name":"ISPRS Journal of Photogrammetry and Remote Sensing","volume":"218 ","pages":"Pages 277-293"},"PeriodicalIF":10.6,"publicationDate":"2024-09-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142270435","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"An automatic procedure for mapping burned areas globally using Sentinel-2 and VIIRS/MODIS active fires in Google Earth Engine","authors":"Aitor Bastarrika ,&nbsp;Armando Rodriguez-Montellano ,&nbsp;Ekhi Roteta ,&nbsp;Stijn Hantson ,&nbsp;Magí Franquesa ,&nbsp;Leyre Torre ,&nbsp;Jon Gonzalez-Ibarzabal ,&nbsp;Karmele Artano ,&nbsp;Pilar Martinez-Blanco ,&nbsp;Amaia Mesanza ,&nbsp;Jesús A. Anaya ,&nbsp;Emilio Chuvieco","doi":"10.1016/j.isprsjprs.2024.08.019","DOIUrl":"10.1016/j.isprsjprs.2024.08.019","url":null,"abstract":"<div><p>Understanding the spatial and temporal trends of burned areas (BA) on a global scale offers a comprehensive view of the underlying mechanisms driving fire incidence and its influence on ecosystems and vegetation recovery patterns over extended periods. Such insights are invaluable for modeling fire emissions and the formulation of strategies for post-fire rehabilitation planning.</p><p>Previous research has provided strong evidence that current global BA products derived from coarse spatial resolution data underestimates global burned areas. Consequently, there is a pressing need for global high-resolution BA products. Here, we present an automatic global burned area mapping algorithm (Sentinel2BAM) based on Sentinel-2 Level-2A imagery combined with Visible Infrared Imaging Radiometer Suite (VIIRS) and Moderate Resolution Imaging Spectrometer (MODIS) active fire data. The algorithm employs a Random Forest Model trained by active fires to predict BA probabilities in each 5-day Normalized Burn Ratio (NBR) index-based temporal composites. In a second step, a time-series and object-based analysis of the estimated BA probabilities allows burned areas to be detected on a quarterly basis. The algorithm was implemented in Google Earth Engine (GEE) and applied to 576 Sentinel-2 tiles corresponding to 2019, distributed globally, to assess its ability to map burned areas across different ecosystems. Two validation sources were employed: 21 EMSR Copernicus Emergency Service perimeters obtained using high spatial resolution (&lt;10 m) data (EMSR21) located in the Mediterranean basin and 50 20x20 km global samples selected by stratified sampling with Sentinel-2 at 10 m spatial resolution (GlobalS50). Additionally, 105 Landsat-based long sample units (GlobalL105), were employed to compare the performance of the Sentinel2BAM algorithm against the FIRECCI51 and MCD64A1 global products. Overall accuracy metrics for the Sentinel2BAM algorithm, derived from validation sources highlight higher commission (CE) than omission (OE) errors (CE=10.3 % and OE=7.6 % when using EMSR21 as reference, CE=18.9 % and OE=9.5 % when using Global S50 as reference), while GlobalL105-based inferenced global comparison metrics show similar patterns (CE=22.5 % and OE=13.4 %). Results indicate differences across ecosystems: forest fires in tropical and temperate biomes exhibit higher CE, mainly due to confusion between burned areas and croplands. According to GlobalL105, Sentinel2BAM shows greater accuracy globally (CE=22.5 %, OE=13.4 %) compared to FIRECCI51 (CE=20.8 %, OE=46.5 %) and MCD64A1 (CE=17.5 %, OE=53.1 %), substantially improving the detection of small fires and thereby reducing omission errors. The strengths and weaknesses of the algorithm are thoroughly addressed, demonstrating its potential for global application.</p></div>","PeriodicalId":50269,"journal":{"name":"ISPRS Journal of Photogrammetry and Remote Sensing","volume":"218 ","pages":"Pages 232-245"},"PeriodicalIF":10.6,"publicationDate":"2024-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142246936","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A thin cloud blind correction method coupling a physical model with unsupervised deep learning for remote sensing imagery","authors":"Liying Xu ,&nbsp;Huifang Li ,&nbsp;Huanfeng Shen ,&nbsp;Chi Zhang ,&nbsp;Liangpei Zhang","doi":"10.1016/j.isprsjprs.2024.09.008","DOIUrl":"10.1016/j.isprsjprs.2024.09.008","url":null,"abstract":"<div><p>Thin cloud disturbs the observation of optical sensors, thus reducing the quality of optical remote sensing images and limiting the subsequent applications. However, the reliance of the existing thin cloud correction methods on the assistance of in-situ parameters, prior assumptions, massive paired data, or special bands severely limits their generalization. Moreover, due to the inadequate consideration of cloud characteristics, these methods struggle to obtain accurate results with complex degradations. To address the above two problems, a thin cloud blind correction (TC-BC) method coupling a cloudy image imaging model and a feature separation network (FSNet) module is proposed in this paper, based on an unsupervised self-training framework. Specifically, the FSNet module takes the independence and obscure boundary characteristics of the cloud into account to improve the correction accuracy with complex degradations. The FSNet module consists of an information interaction structure for exchanging the complementary features between cloud and ground, and a spatially adaptive structure for promoting the learning of the thin cloud distribution. Thin cloud correction experiments were conducted on an unpaired blind correction dataset (UBCSet) and the proposed TC-BC method was compared with three traditional methods. The visual results suggest that the proposed method shows obvious advantages in information recovery for thin cloud cover regions, and shows a superior global consistency between cloudy regions and clear regions. The TC-BC method also achieves the highest peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM). The FSNet module in the TC-BC method is also proven to be effective. The FSNet module can achieve a superior precision when compared with five other deep learning networks in cloud-ground separation performance. Finally, extra experimental results show that the TC-BC method can be applied to different cloud correction scenarios with varied cloud coverage, surface types, and image scales, demonstrating its generalizability. Code: <span><span>https://github.com/Liying-Xu/TCBC</span><svg><path></path></svg></span>.</p></div>","PeriodicalId":50269,"journal":{"name":"ISPRS Journal of Photogrammetry and Remote Sensing","volume":"218 ","pages":"Pages 246-259"},"PeriodicalIF":10.6,"publicationDate":"2024-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142246972","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Nonlinear least-squares solutions to the TLS multi-station registration adjustment problem","authors":"Yu Hu,&nbsp;Xing Fang,&nbsp;Wenxian Zeng","doi":"10.1016/j.isprsjprs.2024.09.014","DOIUrl":"10.1016/j.isprsjprs.2024.09.014","url":null,"abstract":"<div><p>Performing multiple scans is necessary to cover an entire scene of interest, making multi-station registration adjustment a critical task in terrestrial laser scanner data processing. Existing methods either rely on pair-wise adjustment, which leads to drift accumulation and lacks global consistency, or provide an approximate solution based on a linearized model, sacrificing statistical optimality. In this study, using a multi-station stacking model, we propose a method that provides two different nonlinear least-squares (LS) solutions to this problem. We first demonstrate how a nonlinear Baarda’s S-transformation can be used to transform solutions that share the same optimal network configuration. Then, two practically meaningful LS solutions are introduced, i.e., the trivial minimal-constraints solution and the partial nearest solution. Most importantly, we derive a truncated Gauss–Newton iterative scheme to obtain numerically exact solutions to the corresponding nonlinear rank-deficient problem. We validate our method with three real-world examples, demonstrating that (1) global consistency is maintained with no drift accumulation, and (2) our nonlinear solution outperforms the approximate linearized solution. Code and data are available at <span><span>https://github.com/huyuchn/Multi-station-registration</span><svg><path></path></svg></span>.</p></div>","PeriodicalId":50269,"journal":{"name":"ISPRS Journal of Photogrammetry and Remote Sensing","volume":"218 ","pages":"Pages 220-231"},"PeriodicalIF":10.6,"publicationDate":"2024-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142243277","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Joint block adjustment and variational optimization for global and local radiometric normalization toward multiple remote sensing image mosaicking","authors":"Dekun Lin ,&nbsp;Huanfeng Shen ,&nbsp;Xinghua Li ,&nbsp;Chao Zeng ,&nbsp;Tao Jiang ,&nbsp;Yongming Ma ,&nbsp;Mingjie Xu","doi":"10.1016/j.isprsjprs.2024.08.016","DOIUrl":"10.1016/j.isprsjprs.2024.08.016","url":null,"abstract":"<div><p>Multi-temporal optical remote sensing images acquired from cross-sensor platforms often show significant radiometric differences, posing challenges when mosaicking images. These challenges include inconsistent global radiometric tones, unsmooth local radiometric transitions, and visible seamlines. In this paper, to address these challenges, we propose a two-stage approach for global and local radiometric normalization (RN) using joint block adjustment and variational optimization. In the first stage, a block adjustment based global RN (BAGRN) model is established to simultaneously perform global RN on all the images, eliminating global radiometric differences and achieving overall radiometric tonal consistency. In the second stage, a variational optimization based local RN (VOLRN) model is introduced to address the remaining local radiometric differences after global RN. The VOLRN model applies local RN to all the image blocks within a unified energy function and imposes the <span><math><mrow><msub><mi>l</mi><mn>1</mn></msub></mrow></math></span> norm constraint on the data fidelity term, providing the model with a more flexible local RN capability to radiometrically normalize the intersection and transition areas of the images. Therefore, the local radiometric discontinuities and edge artifacts can be eliminated, resulting in natural and smooth local radiometric transitions. The experimental results obtained on five challenging datasets of cross-sensor and multi-temporal remote sensing images demonstrate that the proposed approach excels in both visual quality and quantitative metrics. The proposed approach effectively eliminates global and local radiometric differences, preserves image gradients well, and has high processing efficiency. As a result, it outperforms the state-of-the-art RN approaches.</p></div>","PeriodicalId":50269,"journal":{"name":"ISPRS Journal of Photogrammetry and Remote Sensing","volume":"218 ","pages":"Pages 187-203"},"PeriodicalIF":10.6,"publicationDate":"2024-09-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142243271","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Weather-aware autopilot: Domain generalization for point cloud semantic segmentation in diverse weather scenarios","authors":"Jing Du ,&nbsp;John Zelek ,&nbsp;Jonathan Li","doi":"10.1016/j.isprsjprs.2024.09.006","DOIUrl":"10.1016/j.isprsjprs.2024.09.006","url":null,"abstract":"<div><p>3D point cloud semantic segmentation, a pivotal task in fields such as autonomous driving and urban planning, confronts the challenge of performance degradation under adverse weather conditions. Current methodologies primarily focus on optimal weather scenarios, leaving a significant gap in handling various environmental adversities like fog, rain, and snow. To bridge this gap, we propose a comprehensive deep learning framework featuring unique components — an Adaptive Feature Normalization Module (AFNM) for effective normalization and calibration of features, a Dual-Attention Fusion Module (DAFM) for integrating cross-domain features, and a Proxy Label Generation Module (PLGM) for generating reliable proxy labels within the domain. Utilizing the SemanticKITTI and SynLiDAR datasets as source domains and the SemanticSTF dataset as the target domain, our model has been rigorously evaluated under varying weather conditions. When trained on the SemanticKITTI dataset as the source domain with the SemanticSTF dataset as the target, our approach surpasses the current state-of-the-art models by a margin of 6.2% in terms of overall mean Intersection over Union (mIoU) scores. Similarly, with the SynLiDAR dataset as the source and SemanticSTF as the target, our performance exceeds the best existing models by 3.4% in mIoU. These results substantiate the efficacy of our model in advancing the field of 3D semantic segmentation under diverse weather conditions, showcasing its notable robustness and superiority. The code is available at <span><span>https://github.com/J2DU/WADG-PointSeg</span><svg><path></path></svg></span>.</p></div>","PeriodicalId":50269,"journal":{"name":"ISPRS Journal of Photogrammetry and Remote Sensing","volume":"218 ","pages":"Pages 204-219"},"PeriodicalIF":10.6,"publicationDate":"2024-09-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142243276","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Reconstructing NDVI time series in cloud-prone regions: A fusion-and-fit approach with deep learning residual constraint","authors":"Peng Qin ,&nbsp;Huabing Huang ,&nbsp;Peimin Chen ,&nbsp;Hailong Tang ,&nbsp;Jie Wang ,&nbsp;Shuang Chen","doi":"10.1016/j.isprsjprs.2024.09.010","DOIUrl":"10.1016/j.isprsjprs.2024.09.010","url":null,"abstract":"<div><p>The time series data of Normalized Difference Vegetation Index (NDVI) is crucial for monitoring changes in terrestrial vegetation. Existing reconstruction methods encounter challenges in areas prone to clouds, primarily due to inadequate utilization of spatial, temporal, periodic, and multi-sensor information, as well as a lack of physical interpretations. This frequently results in limited model performance or the omission of spatial details when predicting scenarios involving land cover changes. In this study, we propose a novel approach named Residual (Re) Constraints (Co) fusion-and-fit (ReCoff), consisting of two steps: ReCoF fusion (F) and Savitzky-Golay (SG) fit. This approach addresses the challenges of reconstructing 30 m Landsat NDVI time series data in cloudy regions. The fusion-fit process captures land cover changes and maps them from MODIS to Landsat using a deep learning model with residual constraints, while simultaneously integrating multi-dimensional, multi-sensor, and long time-series information. ReCoff offers three distinct advantages. First, the fusion results are more robust to land cover change scenarios and contain richer spatial details (RMSE of 0.091 vs. 0.101, 0.164, and 0.188 for ReCoF vs. STFGAN, FSDAF, and ESTARFM). Second, ReCoff improves the effectiveness of reconstructing dense time-series data (2016–2020, 16-day interval) in cloudy areas, whereas other methods are more susceptible to the impact of prolonged data gaps. ReCoff achieves a correlation coefficient of 0.84 with the MODIS reference series, outperforming SG (0.28), HANTS (0.32), and GF-SG (0.48). Third, with the help of the GEE platform, ReCoff can be applied over large areas (771 km × 634 km) and long-time scales (bimonthly intervals from 2000 to 2020) in cloudy regions. ReCoff demonstrates potential for accurately reconstructing time-series data in cloudy areas.</p></div>","PeriodicalId":50269,"journal":{"name":"ISPRS Journal of Photogrammetry and Remote Sensing","volume":"218 ","pages":"Pages 170-186"},"PeriodicalIF":10.6,"publicationDate":"2024-09-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142233854","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}