IEEE transactions on pattern analysis and machine intelligence最新文献_第4页

Graph Regulation Network for Point Cloud Segmentation. 用于点云分割的图形调节网络

IEEE transactions on pattern analysis and machine intelligence Pub Date : 2024-12-01 Epub Date: 2024-11-06 DOI: 10.1109/TPAMI.2024.3400402

Zijin Du, Jianqing Liang, Jiye Liang, Kaixuan Yao, Feilong Cao

{"title":"Graph Regulation Network for Point Cloud Segmentation.","authors":"Zijin Du, Jianqing Liang, Jiye Liang, Kaixuan Yao, Feilong Cao","doi":"10.1109/TPAMI.2024.3400402","DOIUrl":"10.1109/TPAMI.2024.3400402","url":null,"abstract":"In point cloud, some regions typically exist nodes from multiple categories, i.e., these regions have both homophilic and heterophilic nodes. However, most existing methods ignore the heterophily of edges during the aggregation of the neighborhood node features, which inevitably mixes unnecessary information of heterophilic nodes and leads to blurred boundaries of segmentation. To address this problem, we model the point cloud as a homophilic-heterophilic graph and propose a graph regulation network (GRN) to produce finer segmentation boundaries. The proposed method can adaptively adjust the propagation mechanism with the degree of neighborhood homophily. Moreover, we build a prototype feature extraction module, which is utilised to mine the homophily features of nodes from the global prototype space. Theoretically, we prove that our convolution operation can constrain the similarity of representations between nodes based on their degree of homophily. Extensive experiments on fully and weakly supervised point cloud semantic segmentation tasks demonstrate that our method achieves satisfactory performance. Especially in the case of weak supervision, that is, each sample has only 1%-10% labeled points, the proposed method has a significant improvement in segmentation performance.","PeriodicalId":94034,"journal":{"name":"IEEE transactions on pattern analysis and machine intelligence","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140917570","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Self-Supervised 3D Scene Flow Estimation and Motion Prediction Using Local Rigidity Prior. 利用局部刚度先验进行自监督三维场景流估计和运动预测

IEEE transactions on pattern analysis and machine intelligence Pub Date : 2024-12-01 Epub Date: 2024-11-06 DOI: 10.1109/TPAMI.2024.3401029

Ruibo Li, Chi Zhang, Zhe Wang, Chunhua Shen, Guosheng Lin

{"title":"Self-Supervised 3D Scene Flow Estimation and Motion Prediction Using Local Rigidity Prior.","authors":"Ruibo Li, Chi Zhang, Zhe Wang, Chunhua Shen, Guosheng Lin","doi":"10.1109/TPAMI.2024.3401029","DOIUrl":"10.1109/TPAMI.2024.3401029","url":null,"abstract":"In this article, we investigate self-supervised 3D scene flow estimation and class-agnostic motion prediction on point clouds. A realistic scene can be well modeled as a collection of rigidly moving parts, therefore its scene flow can be represented as a combination of rigid motion of these individual parts. Building upon this observation, we propose to generate pseudo scene flow labels for self-supervised learning through piecewise rigid motion estimation, in which the source point cloud is decomposed into local regions and each region is treated as rigid. By rigidly aligning each region with its potential counterpart in the target point cloud, we obtain a region-specific rigid transformation to generate its pseudo flow labels. To mitigate the impact of potential outliers on label generation, when solving the rigid registration for each region, we alternately perform three steps: establishing point correspondences, measuring the confidence for the correspondences, and updating the rigid transformation based on the correspondences and their confidence. As a result, confident correspondences will dominate label generation, and a validity mask will be derived for the generated pseudo labels. By using the pseudo labels together with their validity mask for supervision, models can be trained in a self-supervised manner. Extensive experiments on FlyingThings3D and KITTI datasets demonstrate that our method achieves new state-of-the-art performance in self-supervised scene flow learning, without any ground truth scene flow for supervision, even performing better than some supervised counterparts. Additionally, our method is further extended to class-agnostic motion prediction and significantly outperforms previous state-of-the-art self-supervised methods on nuScenes dataset.","PeriodicalId":94034,"journal":{"name":"IEEE transactions on pattern analysis and machine intelligence","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140924155","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Neural Disparity Refinement. 神经差异细化

IEEE transactions on pattern analysis and machine intelligence Pub Date : 2024-12-01 Epub Date: 2024-11-06 DOI: 10.1109/TPAMI.2024.3411292

Fabio Tosi, Filippo Aleotti, Pierluigi Zama Ramirez, Matteo Poggi, Samuele Salti, Stefano Mattoccia, Luigi Di Stefano

引用次数: 0

Revisiting the Trade-Off Between Accuracy and Robustness via Weight Distribution of Filters. 通过过滤器的权重分布重新审视准确性和鲁棒性之间的权衡。

IEEE transactions on pattern analysis and machine intelligence Pub Date : 2024-12-01 Epub Date: 2024-11-06 DOI: 10.1109/TPAMI.2024.3411035

Xingxing Wei, Shiji Zhao, Bo Li

{"title":"Revisiting the Trade-Off Between Accuracy and Robustness via Weight Distribution of Filters.","authors":"Xingxing Wei, Shiji Zhao, Bo Li","doi":"10.1109/TPAMI.2024.3411035","DOIUrl":"10.1109/TPAMI.2024.3411035","url":null,"abstract":"Adversarial attacks have been proven to be potential threats to Deep Neural Networks (DNNs), and many methods are proposed to defend against adversarial attacks. However, while enhancing the robustness, the accuracy for clean examples will decline to a certain extent, implying a trade-off existed between the accuracy and adversarial robustness. In this paper, to meet the trade-off problem, we theoretically explore the underlying reason for the difference of the filters' weight distribution between standard-trained and robust-trained models and then argue that this is an intrinsic property for static neural networks, thus they are difficult to fundamentally improve the accuracy and adversarial robustness at the same time. Based on this analysis, we propose a sample-wise dynamic network architecture named Adversarial Weight-Varied Network (AW-Net), which focuses on dealing with clean and adversarial examples with a \"divide and rule\" weight strategy. The AW-Net adaptively adjusts the network's weights based on regulation signals generated by an adversarial router, which is directly influenced by the input sample. Benefiting from the dynamic network architecture, clean and adversarial examples can be processed with different network weights, which provides the potential to enhance both accuracy and adversarial robustness. A series of experiments demonstrate that our AW-Net is architecture-friendly to handle both clean and adversarial examples and can achieve better trade-off performance than state-of-the-art robust models.","PeriodicalId":94034,"journal":{"name":"IEEE transactions on pattern analysis and machine intelligence","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141289055","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Dual Input Stream Transformer for Vertical Drift Correction in Eye-Tracking Reading Data. 用于眼动跟踪阅读数据垂直漂移校正的双输入流变换器

IEEE transactions on pattern analysis and machine intelligence Pub Date : 2024-12-01 Epub Date: 2024-11-06 DOI: 10.1109/TPAMI.2024.3411938

Thomas M Mercier, Marcin Budka, Martin R Vasilev, Julie A Kirkby, Bernhard Angele, Timothy J Slattery

{"title":"Dual Input Stream Transformer for Vertical Drift Correction in Eye-Tracking Reading Data.","authors":"Thomas M Mercier, Marcin Budka, Martin R Vasilev, Julie A Kirkby, Bernhard Angele, Timothy J Slattery","doi":"10.1109/TPAMI.2024.3411938","DOIUrl":"10.1109/TPAMI.2024.3411938","url":null,"abstract":"We introduce a novel Dual Input Stream Transformer (DIST) for the challenging problem of assigning fixation points from eye-tracking data collected during passage reading to the line of text that the reader was actually focused on. This post-processing step is crucial for analysis of the reading data due to the presence of noise in the form of vertical drift. We evaluate DIST against eleven classical approaches on a comprehensive suite of nine diverse datasets. We demonstrate that combining multiple instances of the DIST model in an ensemble achieves high accuracy across all datasets. Further combining the DIST ensemble with the best classical approach yields an average accuracy of 98.17%. Our approach presents a significant step towards addressing the bottleneck of manual line assignment in reading research. Through extensive analysis and ablation studies, we identify key factors that contribute to DIST's success, including the incorporation of line overlap features and the use of a second input stream. Via rigorous evaluation, we demonstrate that DIST is robust to various experimental setups, making it a safe first choice for practitioners in the field.","PeriodicalId":94034,"journal":{"name":"IEEE transactions on pattern analysis and machine intelligence","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141302310","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Pair Then Relation: Pair-Net for Panoptic Scene Graph Generation. 成对关系：用于全景图生成的 Pair-Net

IEEE transactions on pattern analysis and machine intelligence Pub Date : 2024-12-01 Epub Date: 2024-11-06 DOI: 10.1109/TPAMI.2024.3442301

Jinghao Wang, Zhengyu Wen, Xiangtai Li, Zujin Guo, Jingkang Yang, Ziwei Liu

{"title":"Pair Then Relation: Pair-Net for Panoptic Scene Graph Generation.","authors":"Jinghao Wang, Zhengyu Wen, Xiangtai Li, Zujin Guo, Jingkang Yang, Ziwei Liu","doi":"10.1109/TPAMI.2024.3442301","DOIUrl":"10.1109/TPAMI.2024.3442301","url":null,"abstract":"Panoptic Scene Graph (PSG) is a challenging task in Scene Graph Generation (SGG) that aims to create a more comprehensive scene graph representation using panoptic segmentation instead of boxes. Compared to SGG, PSG has several challenging problems: pixel-level segment outputs and full relationship exploration (It also considers thing and stuff relation). Thus, current PSG methods have limited performance, which hinders downstream tasks or applications. This work aims to design a novel and strong baseline for PSG. To achieve that, we first conduct an in-depth analysis to identify the bottleneck of the current PSG models, finding that inter-object pair-wise recall is a crucial factor that was ignored by previous PSG methods. Based on this and the recent query-based frameworks, we present a novel framework: Pair then Relation (Pair-Net), which uses a Pair Proposal Network (PPN) to learn and filter sparse pair-wise relationships between subjects and objects. Moreover, we also observed the sparse nature of object pairs for both. Motivated by this, we design a lightweight Matrix Learner within the PPN, which directly learns pair-wised relationships for pair proposal generation. Through extensive ablation and analysis, our approach significantly improves upon leveraging the segmenter solid baseline. Notably, our method achieves over 10% absolute gains compared to our baseline, PSGFormer.","PeriodicalId":94034,"journal":{"name":"IEEE transactions on pattern analysis and machine intelligence","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141977458","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Diversifying Policies With Non-Markov Dispersion to Expand the Solution Space. 利用非马尔可夫离散性的多样化政策来扩展求解空间

IEEE transactions on pattern analysis and machine intelligence Pub Date : 2024-12-01 Epub Date: 2024-11-06 DOI: 10.1109/TPAMI.2024.3455257

Bohao Qu, Xiaofeng Cao, Yi Chang, Ivor W Tsang, Yew-Soon Ong

{"title":"Diversifying Policies With Non-Markov Dispersion to Expand the Solution Space.","authors":"Bohao Qu, Xiaofeng Cao, Yi Chang, Ivor W Tsang, Yew-Soon Ong","doi":"10.1109/TPAMI.2024.3455257","DOIUrl":"10.1109/TPAMI.2024.3455257","url":null,"abstract":"Policy diversity, encompassing the variety of policies an agent can adopt, enhances reinforcement learning (RL) success by fostering more robust, adaptable, and innovative problem-solving in the environment. The environment in which standard RL operates is usually modeled with a Markov Decision Process (MDP) as the theoretical foundation. However, in many real-world scenarios, the rewards depend on an agent's history of states and actions leading to a non-MDP. Under the premise of policy diffusion initialization, non-MDPs may have unstructured expanding solution space due to varying historical information and temporal dependencies. This results in solutions having non-equivalent closed forms in non-MDPs. In this paper, deriving diverse solutions for non-MDPs requires policies to break through the boundaries of the current solution space through gradual dispersion. The goal is to expand the solution space, thereby obtaining more diverse policies. Specifically, we first model the sequences of states and actions by a transformer-based method to learn policy embeddings for dispersion in the solution space, since the transformer has advantages in handling sequential data and capturing long-range dependencies for non-MDP. Then, we stack the policy embeddings to construct a dispersion matrix as the policy diversity measure to induce the policy dispersion in the solution space and obtain a set of diverse policies. Finally, we prove that if the dispersion matrix is positive definite, the dispersed embeddings can effectively enlarge the disagreements across policies, yielding a diverse expression for the original policy embedding distribution. Experimental results of both non-MDP and MDP environments show that this dispersion scheme can obtain more expressive diverse policies via expanding the solution space, showing more robust performance than the recent learning baselines.","PeriodicalId":94034,"journal":{"name":"IEEE transactions on pattern analysis and machine intelligence","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142143505","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Biarchetype Analysis: Simultaneous Learning of Observations and Features Based on Extremes. 双车型分析：基于极值同时学习观察结果和特征

IEEE transactions on pattern analysis and machine intelligence Pub Date : 2024-12-01 Epub Date: 2024-11-06 DOI: 10.1109/TPAMI.2024.3400730

Aleix Alcacer, Irene Epifanio, Ximo Gual-Arnau

引用次数: 0

HC ²L: Hybrid and Cooperative Contrastive Learning for Cross-Lingual Spoken Language Understanding. HC2L：用于跨语言口语理解的混合合作对比学习。

IEEE transactions on pattern analysis and machine intelligence Pub Date : 2024-12-01 Epub Date: 2024-11-06 DOI: 10.1109/TPAMI.2024.3402746

Bowen Xing, Ivor W Tsang

{"title":"HC 2L: Hybrid and Cooperative Contrastive Learning for Cross-Lingual Spoken Language Understanding.","authors":"Bowen Xing, Ivor W Tsang","doi":"10.1109/TPAMI.2024.3402746","DOIUrl":"10.1109/TPAMI.2024.3402746","url":null,"abstract":"State-of-the-art model for zero-shot cross-lingual spoken language understanding performs cross-lingual unsupervised contrastive learning to achieve the label-agnostic semantic alignment between each utterance and its code-switched data. However, it ignores the precious intent/slot labels, whose label information is promising to help capture the label-aware semantics structure and then leverage supervised contrastive learning to improve both source and target languages' semantics. In this paper, we propose Hybrid and Cooperative Contrastive Learning to address this problem. Apart from cross-lingual unsupervised contrastive learning, we design a holistic approach that exploits source language supervised contrastive learning, cross-lingual supervised contrastive learning and multilingual supervised contrastive learning to perform label-aware semantics alignments in a comprehensive manner. Each kind of supervised contrastive learning mechanism includes both single-task and joint-task scenarios. In our model, one contrastive learning mechanism's input is enhanced by others. Thus the total four contrastive learning mechanisms are cooperative to learn more consistent and discriminative representations in the virtuous cycle during the training process. Experiments show that our model obtains consistent improvements over 9 languages, achieving new state-of-the-art performance.","PeriodicalId":94034,"journal":{"name":"IEEE transactions on pattern analysis and machine intelligence","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141072477","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Relational Proxies: Fine-Grained Relationships as Zero-Shot Discriminators. 关系代理：细粒度关系作为零误差判别器

IEEE transactions on pattern analysis and machine intelligence Pub Date : 2024-12-01 Epub Date: 2024-11-06 DOI: 10.1109/TPAMI.2024.3408913

Abhra Chaudhuri, Massimiliano Mancini, Zeynep Akata, Anjan Dutta

{"title":"Relational Proxies: Fine-Grained Relationships as Zero-Shot Discriminators.","authors":"Abhra Chaudhuri, Massimiliano Mancini, Zeynep Akata, Anjan Dutta","doi":"10.1109/TPAMI.2024.3408913","DOIUrl":"10.1109/TPAMI.2024.3408913","url":null,"abstract":"Visual categories that largely share the same set of local parts cannot be discriminated based on part information alone, as they mostly differ in the way the local parts relate to the overall global structure of the object. We propose Relational Proxies, a novel approach that leverages the relational information between the global and local views of an object for encoding its semantic label, even for categories it has not encountered during training. Starting with a rigorous formalization of the notion of distinguishability between categories that share attributes, we prove the necessary and sufficient conditions that a model must satisfy in order to learn the underlying decision boundaries to tell them apart. We design Relational Proxies based on our theoretical findings and evaluate it on seven challenging fine-grained benchmark datasets and achieve state-of-the-art results on all of them, surpassing the performance of all existing works with a margin exceeding 4% in some cases. We additionally show that Relational Proxies also generalizes to the zero-shot setting, where it can efficiently leverage emergent relationships among attributes and image views to generalize to unseen categories, surpassing current state-of-the-art in both the non-generative and generative settings. Implementation is available at https://github.com/abhrac/relational-proxies.","PeriodicalId":94034,"journal":{"name":"IEEE transactions on pattern analysis and machine intelligence","volume":null,"pages":null},"PeriodicalIF":0.0,"publicationDate":"2024-12-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141249131","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0