IEEE transactions on image processing : a publication of the IEEE Signal Processing Society最新文献

{"title":"Detector With Classifier2: An End-to-End Multi-Stream Feature Aggregation Network for Fine-Grained Object Detection in Remote Sensing Images","authors":"Shangdong Zheng;Zebin Wu;Yang Xu;Chengxun He;Zhihui Wei","doi":"10.1109/TIP.2025.3563708","DOIUrl":"10.1109/TIP.2025.3563708","url":null,"abstract":"Fine-grained object detection (FGOD) fundamentally comprises two primary tasks: object detection and fine-grained classification. In natural scenes, most FGOD methods benefit from higher instance resolution and fewer environmental variation, attributing more commonly associated with the latter task. In this paper, we propose a unified paradigm named Detector with Classifier2 (DC2), which provides a holistic paradigm by explicitly considering the end-to-end integration of object detection and fine-grained classification tasks, rather than prioritizing one aspect. Initially, our detection sub-network is restricted to only determining whether the proposal is a coarse-category and does not delve into the specific sub-categories. Moreover, in order to reduce redundant pixel-level calculation, we propose an instance-level feature enhancement (IFE) module to model the semantic similarities among proposals, which poses great potential for locating more instances in remote sensing images (RSIs). After obtaining the coarse detection predictions, we further construct a classification sub-network, which is built on top of the former branch to determine the specific sub-categories of the aforementioned predictions. Importantly, the detection network is performed on the complete image, while the classification network conducts secondary modeling for the detected regions. These operations can be denoted as the global contextual information and local intrinsic cues extractions for each instance. Therefore, we propose a multi-stream feature aggregation (MSFA) module to integrate global-stream semantic information and local-stream discriminative cues. Our whole DC2 network follows an end-to-end learning fashion, which effectively excavates the internal correlation between detection and fine-grained classification networks. We evaluate the performance of our DC2 network on two benchmarks SAT-MTB and HRSC2016 datasets. Importantly, our method achieves the new state-of-the-art results compared with recent works (approximately 7% mAP gains on SAT-MTB) and improves baseline by a significant margin (43.2% <inline-formula> <tex-math>$v.s.~36.7$ </tex-math></inline-formula>%) without any complicated post-processing strategies. Source codes of the proposed methods are available at <uri>https://github.com/zhengshangdong/DC2</uri>","PeriodicalId":94032,"journal":{"name":"IEEE transactions on image processing : a publication of the IEEE Signal Processing Society","volume":"34 ","pages":"2707-2720"},"PeriodicalIF":0.0,"publicationDate":"2025-04-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143893504","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Learning to See Low-Light Images via Feature Domain Adaptation","authors":"Qirui Yang;Qihua Cheng;Huanjing Yue;Le Zhang;Yihao Liu;Jingyu Yang","doi":"10.1109/TIP.2025.3563775","DOIUrl":"10.1109/TIP.2025.3563775","url":null,"abstract":"Raw low-light image enhancement (LLIE) has achieved much better performance than the sRGB domain enhancement methods due to the merits of raw data. However, the ambiguity between noisy to clean and raw to sRGB mappings may mislead the single-stage enhancement networks. The two-stage networks avoid ambiguity by step-by-step or decoupling the two mappings but usually have large computing complexity. To solve this problem, we propose a single-stage network empowered by Feature Domain Adaptation (FDA) to decouple the denoising and color mapping tasks in raw LLIE. The denoising encoder is supervised by the clean raw image, and then the denoised features are adapted for the color mapping task by an FDA module. We propose a Lineformer to serve as the FDA, which can well explore the global and local correlations with fewer line buffers (friendly to the line-based imaging process). During inference, the raw supervision branch is removed. In this way, our network combines the advantage of a two-stage enhancement process with the efficiency of single-stage inference. Experiments on four benchmark datasets demonstrate that our method achieves state-of-the-art performance with fewer computing costs (60% FLOPs of the two-stage method DNF). Our codes will be released after the acceptance of this work.","PeriodicalId":94032,"journal":{"name":"IEEE transactions on image processing : a publication of the IEEE Signal Processing Society","volume":"34 ","pages":"2680-2693"},"PeriodicalIF":0.0,"publicationDate":"2025-04-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143889984","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Progressive Invariant Causal Feature Learning for Single Domain Generalization","authors":"Yuxuan Wang;Muli Yang;Aming Wu;Cheng Deng","doi":"10.1109/TIP.2025.3563772","DOIUrl":"10.1109/TIP.2025.3563772","url":null,"abstract":"Single domain generalization (SDG) aims to transfer models trained on a single source domain to multiple unseen target domains while against the unknown domain shifts. The main challenge lies in learning the domain-invariant features to mitigate the domain shift impact. To address this challenge, we reconsider SDG from a causal perspective to capture the domain-invariant features accurately. Specifically, we present a Progressive Invariant Causal Feature Learning (PICF) method that leverages front-door adjustment to gradually obtain the invariant causal features for SDG. First, we introduce a foreground feature filter, which removes object-irrelevant confounders in a cyclical manner to extract the object-related causal features. Subsequently, to further enhance the causal feature invariance, we propose to train with augmented causal features by combining them with randomly-sampled styles from the object-irrelevant feature distribution boundary. As a result, our model bridges the gap between one seen domain and multiple unseen ones by capturing the invariant causal features, which largely enhances the model’s generalization ability in SDG. In experiments, our method can be plugged into multiple state-of-the-art methods, and the significant performance improvements on multiple datasets demonstrate the superiority of our method. In particular, on the PACS dataset, our method achieves an accuracy improvement of 4.7%.","PeriodicalId":94032,"journal":{"name":"IEEE transactions on image processing : a publication of the IEEE Signal Processing Society","volume":"34 ","pages":"2694-2706"},"PeriodicalIF":0.0,"publicationDate":"2025-04-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143890267","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Consistency-Queried Transformer for Audio-Visual Segmentation","authors":"Ying Lv;Zhi Liu;Xiaojun Chang","doi":"10.1109/TIP.2025.3563076","DOIUrl":"10.1109/TIP.2025.3563076","url":null,"abstract":"Audio-visual segmentation (AVS) aims to segment objects in audio-visual content. The effective interaction between audio and visual features has garnered significant attention from the multimodal domain. Despite significant advancements, most existing AVS methods are hampered by multimodal inconsistencies. These inconsistencies primarily manifest as a mismatch between audio and visual information guided by audio cues, wherein visual features often dominate audio modality. To address this issue, we propose the Consistency-Queried Transformer (CQFormer), a novel framework for AVS tasks that leverages the transformer architecture. This framework features a Consistency Query Generator (CQG) and a Query-Aligned Matching (QAM) module. The Noise Contrastive Estimation (NCE) loss function enhances modality matching and consistency by minimizing the distributional differences between audio and visual features, facilitating effective fusion and interaction between these features. Additionally, introducing the consistency query during the decoding stage enhances consistency constraints and object-level semantic information, further improving the accuracy and stability of audio-visual segmentation. Extensive experiments on the popular benchmark of the audio-visual segmentation dataset demonstrate that the proposed CQFormer achieves state-of-the-art performance.","PeriodicalId":94032,"journal":{"name":"IEEE transactions on image processing : a publication of the IEEE Signal Processing Society","volume":"34 ","pages":"2616-2627"},"PeriodicalIF":0.0,"publicationDate":"2025-04-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143884573","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Webly Supervised Fine-Grained Classification by Integrally Tackling Noises and Subtle Differences","authors":"Junjie Chen;Jiebin Yan;Yuming Fang;Li Niu","doi":"10.1109/TIP.2025.3562740","DOIUrl":"10.1109/TIP.2025.3562740","url":null,"abstract":"Webly-supervised fine-grained visual classification (WSL-FGVC) aims to learn similar sub-classes from cheap web images, which suffers from two major issues: label noises in web images and subtle differences among fine-grained classes. However, existing methods for WSL-FGVC only focus on suppressing noise at image-level, but neglect to mine cues at pixel-level to distinguish the subtle differences among fine-grained classes. In this paper, we propose a bag-level top-down attention framework, which could tackle label noises and mine subtle cues simultaneously and integrally. Specifically, our method first extracts high-level semantic information from a bag of images belonging to the same class, and then uses the bag-level information to mine discriminative regions in various scales of each image. Besides, we propose to derive attention weights from attention maps to weight the bag-level fusion for a robust supervision. We also propose an attention loss on self-bag attention and cross-bag attention to facilitate the learning of valid attention. Extensive experiments on four WSL-FGVC datasets, i.e., Web-Aircraft, Web-Bird, Web-Car, and WebiNat-5089, demonstrate the effectiveness of our method against the state-of-the-art methods.","PeriodicalId":94032,"journal":{"name":"IEEE transactions on image processing : a publication of the IEEE Signal Processing Society","volume":"34 ","pages":"2641-2653"},"PeriodicalIF":0.0,"publicationDate":"2025-04-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143876138","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Emergence Model of Perception With Global-Contour Precedence Based on Gestalt Theory and Primary Visual Cortex","authors":"Jingmeng Li;Hui Wei","doi":"10.1109/TIP.2025.3562054","DOIUrl":"10.1109/TIP.2025.3562054","url":null,"abstract":"Perceptual edge grouping is a technique for organizing the cluttered edge pixels into meaningful structures and further serves high-level vision tasks, which has long been a basic and critical task in computer vision. Existing methods usually have a poor performance when coping with the junctions caused by occlusion and noise in natural images. In this paper, we present GPGrouper, a perceptual edge grouping model based on gestalt theory and the primary visual cortex (V1). Different from the existing methods, GPGrouper leverages the edge representation and grouping matrix (ERGM), a functional structure inspired by V1 mechanisms, to represent edges in a way that can effectively reduce grouping errors caused by occlusion between objects. ERGM is trained with natural image contours and further provides a priori guidance for the construction of the edge connection graph (ECG) that is useful to minimize the impact of noise on grouping. In the experiment, we compared GPGrouper and the state-of-the-art (SOTA) method of perceptual grouping on the visual psychology pathfinder challenge. The results demonstrate that GPGrouper outperforms the SOTA method in grouping performance. Furthermore, in the grouping experiments involving line segments with varying lengths detected by the Line Segment Detector (LSD), as well as those involving superpixel segmentation results with significant levels of interfering noise using the SLIC algorithm, GPGrouper was superior to the existing methods in terms of grouping effect and robustness. Moreover, the results of applying the grouping results to the vision tasks objectness demonstrate that GPGrouper can contribute significantly to high-level visual tasks.","PeriodicalId":94032,"journal":{"name":"IEEE transactions on image processing : a publication of the IEEE Signal Processing Society","volume":"34 ","pages":"2721-2736"},"PeriodicalIF":0.0,"publicationDate":"2025-04-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143876137","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Scale-Aware Crowd Counting Network With Annotation Error Modeling","authors":"Yi-Kuan Hsieh;Jun-Wei Hsieh;Xin Li;Yu-Ming Zhang;Yu-Chee Tseng;Ming-Ching Chang","doi":"10.1109/TIP.2025.3555116","DOIUrl":"10.1109/TIP.2025.3555116","url":null,"abstract":"Traditional crowd-counting networks suffer from information loss when feature maps are reduced by pooling layers, leading to inaccuracies in counting crowds at a distance. Existing methods often assume correct annotations during training, disregarding the impact of noisy annotations, especially in crowded scenes. Furthermore, using a fixed Gaussian density model does not account for the varying pixel distribution of the camera distance. To overcome these challenges, we propose a Scale-Aware Crowd Counting Network (SACC-Net) that introduces a scale-aware loss function with error-compensation capabilities of noisy annotations. For the first time, we simultaneously model labeling errors (mean) and scale variations (variance) by spatially varying Gaussian distributions to produce fine-grained density maps for crowd counting. Furthermore, the proposed scale-aware Gaussian density model can be dynamically approximated with a low-rank approximation, leading to improved convergence efficiency with comparable accuracy. To create a smoother scale-aware feature space, this paper proposes a novel Synthetic Fusion Module (SFM) and an Intra-block Fusion Module (IFM) to generate fine-grained heat maps for better crowd counting. The lightweight version of our model, named SACC-LW, enhances the computational efficiency while retaining accuracy. The superiority and generalization properties of scale-aware loss function are extensively evaluated for different backbone architectures and performance metrics on six public datasets: UCF-QNRF, UCF CC 50, NWPU, ShanghaiTech A, ShanghaiTech B, and JHU. Experimental results also demonstrate that SACC-Net outperforms all state-of-the-art methods, validating its effectiveness in achieving superior crowd-counting accuracy. The source code is available at <uri>https://github.com/Naughty725</uri>.","PeriodicalId":94032,"journal":{"name":"IEEE transactions on image processing : a publication of the IEEE Signal Processing Society","volume":"34 ","pages":"2750-2764"},"PeriodicalIF":0.0,"publicationDate":"2025-04-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143872983","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Parameter-Free Deep Multi-Modal Clustering With Reliable Contrastive Learning","authors":"Zhengzheng Lou;Hang Xue;Yanzheng Wang;Chaoyang Zhang;Xin Yang;Shizhe Hu","doi":"10.1109/TIP.2025.3562083","DOIUrl":"10.1109/TIP.2025.3562083","url":null,"abstract":"Deep multi-modal clustering (DMC) expects to improve clustering performance by exploiting abundant information available from multiple modalities. However, different modalities usually have heterogeneous distribution with uneven quality. This may lead to limited performance, especially for contrastive multi-modal clustering, which inevitably performs contrastive learning between high-quality and low-quality modalities. To tackle this challenge, we propose a novel framework named parameter-free deep multi-modal clustering with reliable contrastive learning (PDMC-RCL). Specifically, the reliable contrastive learning quantifies the relationship between contrastive modality pairs with weight values that will promote the discriminative features learning from useful modality pairs and slow down or even prevent the learning from unreliable modality pairs. Moreover, the reliable contrastive learning is imposed simultaneously at both the feature-level and cluster-level in this framework so that the feature representation learning can benefit from multi-level contrastive learning. It is worth noting that our PDMC-RCL method is parameter-free, which can achieve promising performance without additional hyperparameter tuning. Experimental results on various datasets show the effectiveness of our method over typical state-of-the-art compared DMCs. The source code is available on <uri>https://github.com/ShizheHu</uri>","PeriodicalId":94032,"journal":{"name":"IEEE transactions on image processing : a publication of the IEEE Signal Processing Society","volume":"34 ","pages":"2628-2640"},"PeriodicalIF":0.0,"publicationDate":"2025-04-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143873083","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Dual-Level Modality De-Biasing for RGB-T Tracking","authors":"Yufan Hu;Zekai Shao;Bin Fan;Hongmin Liu","doi":"10.1109/TIP.2025.3562077","DOIUrl":"10.1109/TIP.2025.3562077","url":null,"abstract":"RGB-T tracking aims to effectively leverage the complement ability of visual (RGB) and infrared (TIR) modalities to achieve robust tracking performance in various scenarios. Existing RGB-T tracking methods typically adopt backbone networks pre-trained on large-scale RGB datasets, which can lead to a predisposition toward RGB image patterns. RGB and TIR modalities also exhibit inconsistent responses to regions with diverse properties, resulting in imbalances in tracking decisions. We refer to these issues as feature-level and decision-level biases in the TIR modality. In this paper, we propose a novel dual-level modality de-biasing framework for RGB-T tracking to eliminate the inherent feature and decision-level biases. Specifically, we propose a joint infrared-fusion adapter, comprising an infrared-aware adapter and a cross-fusion adapter, designed to adaptively mitigate feature-level biases and utilize complementary information between the two modalities. In addition to implicit feature-level adjustment, we propose a response-decoupled distillation strategy to explicitly alleviate decision-level biases, aiming to achieve consistently accurate decision-making between the RGB and TIR modalities. Extensive experiments on several popular RGB-T tracking benchmarks validate the effectiveness of our proposed method.","PeriodicalId":94032,"journal":{"name":"IEEE transactions on image processing : a publication of the IEEE Signal Processing Society","volume":"34 ","pages":"2667-2679"},"PeriodicalIF":0.0,"publicationDate":"2025-04-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143866836","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Ray-Aided Quadruple Affiliation Network for Calculating Tumor-Stroma Ratios in Breast Cancers","authors":"Kunping Yang;Linying Chen;Xi Zheng;Xuanping Li;Junhui Lan;Yi Wu;Julia Y. S. Tsang;Gary M. Tse","doi":"10.1109/TIP.2025.3561679","DOIUrl":"10.1109/TIP.2025.3561679","url":null,"abstract":"Tumor-stroma ratio (TSR), which is the area ratio between two components within tumor beds, namely tumor cells and tumor stroma, has been suggested as a promising prognostic feature in breast cancers. However, due to imperfect datasets, and the similarity between tumor stroma and non-tumor stroma, previous algorithms struggle to delineate tumor beds, especially those of histomorphologies with a fibrotic focus. To overcome these limitations, we propose a novel ray-aided quadruple affiliation network (RQA-Net) for calculating TSRs in breast cancers. RQA-Net uses quadruple branches to segment tumor cells and tumor beds simultaneously, where a crisscross task subtraction module (CTS-Module) is designed to locate tumor stroma, grounded on its affiliation relationships with tumor beds. Moreover, we propose an affiliation loss (Aff-Loss) to force identified tumor beds to incorporate tumor cells to enhance their affiliation relationships. Furthermore, we propose a ray-based hypothesis testing (RH-Testing) to obtain line segments from ray equations in tumor beds that can decorate identified tumor beds by overlapping. In summary, RQA-Net precisely predicts tumor cells and tumor beds, and thus supports the calculation of TSRs. We also create a cancerous dataset (CrD-Set) containing 100 slides with an average resolution of <inline-formula> <tex-math>$50,000times 50,000$ </tex-math></inline-formula> pixels from real breast cancer cases, which is the first dataset with pixel-wise tumor bed annotations. Experimental results on existing datasets and CrD-Set demonstrate that compared with previous methods, RQA-Net better calculates breast cancer TSRs by precisely identifying tumor cells and tumor beds. The created CrD-Set and codes in this work will be available online at <uri>https://github.com/Kunpingyang1992/Breast-Cancer-TSR-Calculation</uri>","PeriodicalId":94032,"journal":{"name":"IEEE transactions on image processing : a publication of the IEEE Signal Processing Society","volume":"34 ","pages":"2811-2825"},"PeriodicalIF":0.0,"publicationDate":"2025-04-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143866984","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}