IEEE transactions on image processing : a publication of the IEEE Signal Processing Society最新文献

Crowded Video Individual Counting Informed by Social Grouping and Spatial-Temporal Displacement Priors. 基于社会分组和时空位移先验的拥挤视频个体计数。

IF 13.7

IEEE transactions on image processing : a publication of the IEEE Signal Processing Society Pub Date : 2026-10-19 DOI: 10.1109/TIP.2026.3737046

Hao Lu, Xuhui Zhu, Wenjing Zhang, Yanan Li, Xiang Bai

{"title":"Crowded Video Individual Counting Informed by Social Grouping and Spatial-Temporal Displacement Priors.","authors":"Hao Lu, Xuhui Zhu, Wenjing Zhang, Yanan Li, Xiang Bai","doi":"10.1109/TIP.2026.3737046","DOIUrl":"https://doi.org/10.1109/TIP.2026.3737046","url":null,"abstract":"Video Individual Counting (VIC) is a recently introduced task aiming to estimate foot traffic from a video. It extends Video Crowd Counting (VCC) beyond the per-frame pedestrian count. In contrast to VCC that learns to count pedestrians across frames, VIC must identify co-existent pedestrians between frames, which turns out to be a correspondence problem. Existing VIC approaches, however, can underperform in congested scenes such as metro commuting. To address this, we build WuhanMetroCrowd, one of the first VIC datasets that characterize crowded, dynamic pedestrian flows. It features sparse-to-dense density levels, short-to-long video clips, slow-to-fast flow variations, front-to-back appearance changes, and light-to-heavy occlusions. To better adapt VIC approaches to crowds, we rethink the nature of VIC and recognize two informative priors: i) the social grouping prior that indicates pedestrians tend to gather in groups and ii) the spatial-temporal displacement prior that informs an individual cannot teleport physically. The former inspires us to relax the standard one-to-one (O2O) matching used by VIC to one-to-many (O2M) matching, implemented by an implicit context generator and an O2M matcher; the latter facilitates the design of a displacement prior injector, which strengthens not only O2M matching but also feature extraction and model training. These designs jointly form a novel and strong VIC baseline OMAN++. Extensive experiments show that OMAN++ not only outperforms state-of-the-art VIC baselines on the standard SenseCrowd, CroHD, and MovingDroneCrowd benchmarks, but also indicates a clear advantage in crowded scenes, with a 38.12% error reduction on ourWuhanMetroCrowd dataset. Code, data, and pretrained models are available at https://github.com/tiny-smart/OMAN.","PeriodicalId":94032,"journal":{"name":"IEEE transactions on image processing : a publication of the IEEE Signal Processing Society","volume":"PP ","pages":""},"PeriodicalIF":13.7,"publicationDate":"2026-10-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149225048","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Recurrent Prompting With MambaFormer for Domain Generalized Few-Shot Hyperspectral Image Classification. 基于MambaFormer的域广义少镜头高光谱图像分类循环提示。

IF 13.7

IEEE transactions on image processing : a publication of the IEEE Signal Processing Society Pub Date : 2026-10-19 DOI: 10.1109/TIP.2026.3742663

Bobo Xi, Yun Zhang, Jiaojiao Li, Yunsong Li, Zan Li, Jocelyn Chanussot

{"title":"Recurrent Prompting With MambaFormer for Domain Generalized Few-Shot Hyperspectral Image Classification.","authors":"Bobo Xi, Yun Zhang, Jiaojiao Li, Yunsong Li, Zan Li, Jocelyn Chanussot","doi":"10.1109/TIP.2026.3742663","DOIUrl":"https://doi.org/10.1109/TIP.2026.3742663","url":null,"abstract":"Cross-domain few-shot learning enables the transfer of knowledge from the source domain (SD) to the target domain (TD) with a limited number of annotated samples, proving highly effective in hyperspectral image (HSI) classification. Most existing methods involve iterative training between the SD and TD to achieve few-shot domain adaptation. However, the domain gap still compromises the overall performance of the model on the TD. Additionally, compared to the transformer criticized for speed and memory consumption, the Mamba framework has emerged as a promising alternative with comparable long-distance modeling capabilities while being simplified to linear computational complexity. To this end, we propose a recurrent prompting approach using a Mamba-transformer network (MambaFormer) for domain generalized few-shot HSI classification, referred to as ReProM. Specifically, we develop a novel domain-switch meta-learning strategy (DSMLS) that incorporates cross-domain scenarios into the training process through a \"fast switching\" manner. This strategy includes a domain-specific prompter designed to prevent the model from overfitting to the current domain and to mitigate catastrophic forgetting of knowledge from previous domains, thereby enhancing the generalization ability of the model. To lower the computational demands of the vanilla transformer, the devised MambaFormer integrates the bidirectional Mamba (BiMamba) and transformer architectures to effectively capture essential spectral-spatial information. Comprehensive experiments conducted on three benchmark HSI datasets reveal that the proposed ReProM achieves better generalization than state-of-the-art methods. The source code is available at https://github.com/B-Xi/TIP_2026_ReProM.","PeriodicalId":94032,"journal":{"name":"IEEE transactions on image processing : a publication of the IEEE Signal Processing Society","volume":"PP ","pages":""},"PeriodicalIF":13.7,"publicationDate":"2026-10-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149225101","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

TFGNet: Target Face Generation from Low-Quality Images via Textual Guidance. TFGNet：通过文本引导从低质量图像中生成目标人脸。

IF 13.7

IEEE transactions on image processing : a publication of the IEEE Signal Processing Society Pub Date : 2026-10-16 DOI: 10.1109/TIP.2026.3742673

Xin Wen, Zhiyuan Zha, Shigang Wang, Enguang Zuo, Bihan Wen, Xin Yuan, Jiantao Zhou, Ce Zhu

{"title":"TFGNet: Target Face Generation from Low-Quality Images via Textual Guidance.","authors":"Xin Wen, Zhiyuan Zha, Shigang Wang, Enguang Zuo, Bihan Wen, Xin Yuan, Jiantao Zhou, Ce Zhu","doi":"10.1109/TIP.2026.3742673","DOIUrl":"10.1109/TIP.2026.3742673","url":null,"abstract":"Recent advances indicate that text-guided face generation has attracted considerable attention in the field of computer vision. However, most existing methods assume high-quality (HQ) face inputs, while how to generate HQ and identity-preserving faces from low-quality (LQ) images is still an open problem. In this paper, we propose a novel face generation approach, termed TFGNet, which synthesizes HQ face images from LQ inputs guided by textual descriptions. Unlike most existing methods that rely on HQ inputs, TFGNet leverages external textual descriptions as semantic guidance to directly generate HQ and identity-preserving faces from degraded images. First, we design a unified framework that integrates a Transformer-based encoder, a codebook mechanism, and multimodal representations extracted from contrastive language-image pretraining (CLIP) model to produce enhanced cross-modal embeddings, which are then decoded by a diffusion model to generate target face images with both high visual fidelity and accurate identity (ID) preservation. Second, we propose a masked diffusion loss that emphasizes identity-related regions and embed it into a dynamically weighted total loss, enabling a balanced trade-off among visual fidelity, semantic coherence, and ID consistency. Third, we construct a multimodal dataset comprising LQ face images, HQ targets, and manually annotated textual descriptions to address the scarcity of suitable text-image pairs for this task. Extensive experimental results demonstrate that the proposed TFGNet approach outperforms many state-of-the-art face generation methods in terms of both objective metrics and perceptual quality. The source code of this work is available at: https://github.com/wenxin-cv/TFGNet.","PeriodicalId":94032,"journal":{"name":"IEEE transactions on image processing : a publication of the IEEE Signal Processing Society","volume":"PP ","pages":""},"PeriodicalIF":13.7,"publicationDate":"2026-10-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149208386","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Learning to Reorder: Value-Driven State Space Models for Underwater Image Enhancement. 学习重新排序：水下图像增强的价值驱动状态空间模型。

IF 13.7

IEEE transactions on image processing : a publication of the IEEE Signal Processing Society Pub Date : 2026-10-16 DOI: 10.1109/TIP.2026.3742630

Kui Jiang, Yan Luo, Junjun Jiang, Ke Gu, Nan Ma, Xianming Liu

{"title":"Learning to Reorder: Value-Driven State Space Models for Underwater Image Enhancement.","authors":"Kui Jiang, Yan Luo, Junjun Jiang, Ke Gu, Nan Ma, Xianming Liu","doi":"10.1109/TIP.2026.3742630","DOIUrl":"10.1109/TIP.2026.3742630","url":null,"abstract":"State Space Models (SSMs) have emerged as a compelling backbone for vision tasks, offering linear complexity and global receptive fields. However, their application to Underwater Image Enhancement (UIE) is fundamentally constrained by the inherent mismatch between the standard sequential scanning mechanism and the unique statistical distribution of underwater scenes. Specifically, the dominance of large-scale, homogeneous background regions tends to overshadow the feature responses of sparse yet critical targets during state propagation, leading to the attenuation of semantic information and compromised structural reconstruction. To transcend this limitation, we propose VDSNet, a novel Value-Driven State Space Model framework. The core innovation lies in the Multi-Granularity Value Guidance Learning (MVGL) module, which generates a pixel-aligned value map to dynamically reorganize the scanning sequence, thereby prioritizing informative regions to ensure robust long-range state propagation for salient features. Complementing this, we design a Mamba-Conv Mixer (MCM) block that synergistically integrates priority-driven global sequencing with dynamic local convolutions, effectively balancing background suppression with target enhancement. Additionally, a Cross-Feature Bridge (CFB) is introduced to refine multi-level feature fusion. Extensive experiments demonstrate that VDSNet establishes a new state-of-the-art, surpassing WMamba by 0.89 dB on average while exhibiting superior water bias suppression and color fidelity. Furthermore, a lightweight variant, VDSNet-S, is constructed via efficient operators and resolution rescaling, offering a practical solution for real-time applications.","PeriodicalId":94032,"journal":{"name":"IEEE transactions on image processing : a publication of the IEEE Signal Processing Society","volume":"PP ","pages":""},"PeriodicalIF":13.7,"publicationDate":"2026-10-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149208365","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Implicit-Explicit Decoupling and Mixture-of-Experts Fusion for Subject-Driven Text-to-Image Generation. 主题驱动文本到图像生成的隐式-显式解耦和专家混合融合。

IF 13.7

IEEE transactions on image processing : a publication of the IEEE Signal Processing Society Pub Date : 2026-10-14 DOI: 10.1109/TIP.2026.3741659

Wenqi Ren, Kewen Chen, Xiaobin Hu, Jianhou Gan, Xiaocheng Zhang, Shangfeng Guo, Guoli Yang

{"title":"Implicit-Explicit Decoupling and Mixture-of-Experts Fusion for Subject-Driven Text-to-Image Generation.","authors":"Wenqi Ren, Kewen Chen, Xiaobin Hu, Jianhou Gan, Xiaocheng Zhang, Shangfeng Guo, Guoli Yang","doi":"10.1109/TIP.2026.3741659","DOIUrl":"10.1109/TIP.2026.3741659","url":null,"abstract":"Recent advances in large-scale text-to-image (T2I) models have spurred the development of subject-driven T2I generation, which aims to synthesize customized images that align with textual prompts while preserving subject identity. However, existing approaches still struggle to disentangle identity-related information from identity-irrelevant details, leading to overfitting or inadequate preservation of subject identity. In this work, we propose a novel method that enhances the separation of identity-related and identity-irrelevant features, and introduces a feature fusion mechanism to improve image quality and text alignment. Our method consists of two key components: an implicit-explicit foreground-background decoupling module (IEDM) and a feature fusion module (FFM) based on a Mixture of Experts. IEDM uses a learnable adapter for implicit feature-level decoupling and an inpainting technique for explicit foreground-background separation. This decoupling process is facilitated by three complementary loss functions. FFM adaptively integrates identity-irrelevant and identity-related features, enabling more refined feature representations. Extensive experiments demonstrate the effectiveness of our method in improving image quality, enhancing scene adaptability, and increasing the diversity of outputs across various textual prompts.","PeriodicalId":94032,"journal":{"name":"IEEE transactions on image processing : a publication of the IEEE Signal Processing Society","volume":"PP ","pages":""},"PeriodicalIF":13.7,"publicationDate":"2026-10-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149187093","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Physics-Constrained Diffusion for Underwater Image Restoration with Spatially Varying Background Light. 空间变化背景光下水下图像恢复的物理约束扩散。

IF 13.7

IEEE transactions on image processing : a publication of the IEEE Signal Processing Society Pub Date : 2026-10-09 DOI: 10.1109/TIP.2026.3740356

Huilin Ge, Cheng Fang, Jie Zhu, Qiucen Li, Xiaoping Chen, Jianhou Gan, Wenqi Ren

{"title":"Physics-Constrained Diffusion for Underwater Image Restoration with Spatially Varying Background Light.","authors":"Huilin Ge, Cheng Fang, Jie Zhu, Qiucen Li, Xiaoping Chen, Jianhou Gan, Wenqi Ren","doi":"10.1109/TIP.2026.3740356","DOIUrl":"10.1109/TIP.2026.3740356","url":null,"abstract":"Underwater image restoration is challenging due to light absorption and scattering, which cause color distortion, low contrast, and uneven illumination. Existing methods either ignore the underwater imaging process or rely on simplified assumptions, especially treating background light as a spatially uniform component, which limits restoration performance in complex environments. Recent diffusion-based approaches improve restoration quality by exploiting strong image priors, but their physical modeling of background illumination remains insufficient. In this paper, we propose a physics-constrained diffusion method with spatially varying background light for underwater image restoration. The proposed method explicitly models background light by decomposing it into illumination intensity and mixed chromaticity components, and incorporates physical consistency constraints into the diffusion sampling process. A staged optimization strategy is further introduced to progressively optimize physical parameters while keeping the diffusion prior fixed during inference. Experiments on multiple underwater datasets demonstrate consistent improvements in both full-reference and no-reference metrics. The current implementation requires approximately 568 s per 256×256 image and is therefore more suitable for high-quality offline restoration than real-time applications.","PeriodicalId":94032,"journal":{"name":"IEEE transactions on image processing : a publication of the IEEE Signal Processing Society","volume":"PP ","pages":""},"PeriodicalIF":13.7,"publicationDate":"2026-10-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149149851","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Local Feature Enhancement of dino.txt for Training-Free Open-Vocabulary Semantic Segmentation. 面向开放词汇语义分割的局部特征增强。

IF 13.7

IEEE transactions on image processing : a publication of the IEEE Signal Processing Society Pub Date : 2026-10-09 DOI: 10.1109/TIP.2026.3740308

Cheng He, Shiyu Xuan, Zechao Li

{"title":"Local Feature Enhancement of dino.txt for Training-Free Open-Vocabulary Semantic Segmentation.","authors":"Cheng He, Shiyu Xuan, Zechao Li","doi":"10.1109/TIP.2026.3740308","DOIUrl":"10.1109/TIP.2026.3740308","url":null,"abstract":"Open-vocabulary semantic segmentation aims to establish dense pixel-wise correspondences between images and open-ended semantic categories. While most existing methods build upon CLIP's image-text alignment capability, they require substantial architectural modifications to adapt image-level understanding to pixel-level prediction. The recently introduced dino.txt model offers an alternative by employing DINOv2 as its visual backbone, which inherently captures both image-level and pixel-level representations. Through global and local image-text alignment training, dino.txt learns to establish open-vocabulary correspondences, opening up new possibilities for semantic segmentation. To fully exploit dino.txt for this task, we propose FreeD, a training-free approach that enhances pixel-text matching accuracy while preserving global semantics. FreeD achieves this by adaptively refining intermediate features and attention mechanisms based on token-level local similarities. We further introduce FreeDP, which fuses complementary features from multiple vision-language models to achieve more robust segmentation. Extensive experiments on standard benchmarks demonstrate the state-of-the-art performance of our training-free method.","PeriodicalId":94032,"journal":{"name":"IEEE transactions on image processing : a publication of the IEEE Signal Processing Society","volume":"PP ","pages":""},"PeriodicalIF":13.7,"publicationDate":"2026-10-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149149808","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

EmoICL: Enhancing Visual Emotion Recognition with Emotion-Aware In-Context Learning. EmoICL：通过情境学习增强视觉情绪识别。

IF 13.7

IEEE transactions on image processing : a publication of the IEEE Signal Processing Society Pub Date : 2026-10-09 DOI: 10.1109/TIP.2026.3740270

Xiyang Li, Qu Yang, Yuwendong Yin, He Li, Mang Ye

{"title":"EmoICL: Enhancing Visual Emotion Recognition with Emotion-Aware In-Context Learning.","authors":"Xiyang Li, Qu Yang, Yuwendong Yin, He Li, Mang Ye","doi":"10.1109/TIP.2026.3740270","DOIUrl":"10.1109/TIP.2026.3740270","url":null,"abstract":"Visual emotion recognition aims to infer human emotional states from visual cues. Adapting pretrained Vision-Language Models (VLMs) to this task remains challenging. In-Context Learning (ICL) enables a frozen model to condition its predictions on a few labeled demonstrations without task-specific fine-tuning. Here, demonstrations are retrieved from an existing labeled support pool; the small prompt size does not imply a small total annotation budget. Despite this potential, how to effectively leverage ICL for visual emotion recognition remains underexplored. In this paper, we propose EmoICL, a hybrid retrieval framework that leverages VLMs to enhance emotion recognition. To this end, we first introduce Emotion-Aware Dual Retrieval (EDR), which generates emotion-sensitive descriptions using guided prompts and refines retrieved samples based on visual similarity and cross-modal relevance. Then, we design Category-Aware Diversity Optimization (CDO), which combines category-aware filtering with Maximum Marginal Relevance to reduce label-level interference and improve visual diversity. Experiments on three benchmark datasets show that EmoICL achieves the best average performance across different VLM backbones, with a mean absolute gain of approximately 1.96 percentage points over the strongest baseline by average accuracy for each backbone. Code is available at: https://github.com/Atlantic4Li/EmoICL.","PeriodicalId":94032,"journal":{"name":"IEEE transactions on image processing : a publication of the IEEE Signal Processing Society","volume":"PP ","pages":""},"PeriodicalIF":13.7,"publicationDate":"2026-10-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149148968","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

MotionDreamer: Generating Continuous Intermediate Motions for Complex Motion Interpolation. 为复杂运动插值生成连续的中间运动。

IF 13.7

IEEE transactions on image processing : a publication of the IEEE Signal Processing Society Pub Date : 2026-10-08 DOI: 10.1109/TIP.2026.3739604

Yongrui Ma, Shijie Zhao, Mingde Yao, Junlin Li, Li Zhang, Xiaohong Liu, Qi Dou, Jinwei Gu, Tianfan Xue

{"title":"MotionDreamer: Generating Continuous Intermediate Motions for Complex Motion Interpolation.","authors":"Yongrui Ma, Shijie Zhao, Mingde Yao, Junlin Li, Li Zhang, Xiaohong Liu, Qi Dou, Jinwei Gu, Tianfan Xue","doi":"10.1109/TIP.2026.3739604","DOIUrl":"https://doi.org/10.1109/TIP.2026.3739604","url":null,"abstract":"Interpolating complex motions is challenging due to their nonlinear, non-rigid nature and ambiguities. Recent diffusion-based methods mitigate this challenge as a conditional generation task, optionally with linear motion control. However, they cannot handle these complex motions because their reconstruction training objective biases models toward per-frame appearance fidelity, which can lead to unnatural movement. To address this, we propose MotionDreamer, a novel framework that instills effective motion priors for all intermediate frames to video generators by regularizing diffusion training with ground truth continuous intermediate motions. The key idea is that the continuous intermediate motion can decompose large, complex, and ambiguous motions between specific intermediate and input frames into small, easy-to-estimate components, and thus can serve as a motion prior to represent motions across frames. MotionDreamer then recursively fuses estimated continuous motions for interpolation flows, which only updates a flow offset to the previous estimation. Thus, MotionDreamer interpolates complex motions with superior visual quality, being selected as the best-performing method in 75.1% of the human-study trials. To maximize the performance of MotionDreamer, we collect the first open-source large-scale dataset, VFIVid, for complex motion interpolation. The VFIVid contains over one million training and 130 evaluation high-quality clips with diverse objects and motions, such as dancing, skating, and camera movements. We will release all code and the dataset upon publication.","PeriodicalId":94032,"journal":{"name":"IEEE transactions on image processing : a publication of the IEEE Signal Processing Society","volume":"PP ","pages":""},"PeriodicalIF":13.7,"publicationDate":"2026-10-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149139444","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Test Time Adaptation with An Explicit Geometric Bridge for 3D Point Cloud Segmentation. 基于显式几何桥的测试时间自适应三维点云分割。

IF 13.7

IEEE transactions on image processing : a publication of the IEEE Signal Processing Society Pub Date : 2026-10-07 DOI: 10.1109/TIP.2026.3739627

Shuofeng Sun, Jiwen Lu, Haibin Yan

{"title":"Test Time Adaptation with An Explicit Geometric Bridge for 3D Point Cloud Segmentation.","authors":"Shuofeng Sun, Jiwen Lu, Haibin Yan","doi":"10.1109/TIP.2026.3739627","DOIUrl":"10.1109/TIP.2026.3739627","url":null,"abstract":"Test Time Adaptation (TTA) is a flexible unsupervised domain generalization method that allows models to adapt to the target domain distribution without requiring access to the source domain data and target domain labels. However, when applied to 3D point cloud scene segmentation, the significant domain gaps introduced by complex scenes and the common issue of class imbalance in segmentation tasks can lead to training instabilities in previous TTA methods, resulting in a degradation of the performance of the source model. To address this issue, we consider that explicit geometric features in the target domain can serve as an intermediate bridge that roughly approximates the target domain to provide strong geometric prior of the target domain data distribution, thereby reducing the domain gap and enhancing domain adaptation performance. Therefore, we propose TTA-EGB that uses Explicit Geometric feature as the Bridge to stabilize the Test Time Adaption process. Specifically, in TTA-EGB, we first introduce a non-parametric geometry model to extract explicit geometric feature and use knowledge distillation to guide the source model to stably adapt to the target domain. Furthermore, previous TTA methods often employ a class-agnostic sampling mechanism to select easy samples for optimization, which causes the model to be overly confident in the major classes and neglect minority classes. Therefore, we propose a category-balanced sampling mechanism that dynamically defines sampling thresholds based on class frequencies, which results in a similar proportion of classes in both easy and hard samples, thereby improving the issue of class imbalance. We conduct the main experiments on the more challenge sim-to-real benchmark about synthetic dataset 3DFRONT and the real-world datasets ScanNet and S3DIS for 3D segmentation task. Results show that our method can efficiently improve the mIOU by over 3% on 3DFRONT→ ScanNet and 7% on 3DFRONT→ S3DIS.","PeriodicalId":94032,"journal":{"name":"IEEE transactions on image processing : a publication of the IEEE Signal Processing Society","volume":"PP ","pages":""},"PeriodicalIF":13.7,"publicationDate":"2026-10-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"149129797","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0