Neural Networks最新文献_第6页

Adaptive receptive field graph neural networks 自适应感受野图神经网络

IF 6 1区计算机科学

Neural Networks Pub Date : 2025-06-05 DOI: 10.1016/j.neunet.2025.107658

Hepeng Gao , Funing Yang , Yongjian Yang , Yuanbo Xu , Yijun Su

{"title":"Adaptive receptive field graph neural networks","authors":"Hepeng Gao , Funing Yang , Yongjian Yang , Yuanbo Xu , Yijun Su","doi":"10.1016/j.neunet.2025.107658","DOIUrl":"10.1016/j.neunet.2025.107658","url":null,"abstract":"<div><div>Graph Neural Networks (GNNs) have drawn increasing attention in recent years and achieved outstanding success in many scenarios and tasks. However, existing methods indicate that the performance of representation learning drops dramatically as GNNs deepen, which is attributed to <strong>over-smoothing representation</strong>. To handle the above issue, we propose an adaptive receptive field graph neural network (ADRP-GNN) that aggregates information by adaptively expanding receptive fields with a monolayer graph convolution layer, avoiding deepening to result in the over-smoothing issue. Specifically, we first present a Multi-hop Graph Convolution Network (MuGC) that captures the information of the nodes and their multi-hop neighbors with only one layer, preventing frequent passing messages between nodes from the over-smoothing issue. Then, we design a Meta Learner that realizes the adaptive receptive field for each node to select related neighbor information. Finally, a Backbone Network is employed to enhance the architecture’s learning ability. In addition, our architecture adaptively generates receptive fields instead of handcrafting stacked layers, which can integrate existing GNN frameworks to fit various scenarios. Extensive experiments indicate that our architecture is effective for the over-smoothing issue and improves accuracy by 0.52% to 6.88% compared to state-of-the-art methods on node classification tasks on eight datasets.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"190 ","pages":"Article 107658"},"PeriodicalIF":6.0,"publicationDate":"2025-06-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144253457","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Differentially Private Multimodal Laplacian Dropout (DP-MLD) for EEG representative learning 脑电代表性学习的差分私有多模态拉普拉斯Dropout （DP-MLD）

IF 6 1区计算机科学

Neural Networks Pub Date : 2025-06-04 DOI: 10.1016/j.neunet.2025.107657

Xiaowen Fu , Bingxin Wang , Xinzhou Guo , Guoqing Liu , Yang Xiang

{"title":"Differentially Private Multimodal Laplacian Dropout (DP-MLD) for EEG representative learning","authors":"Xiaowen Fu , Bingxin Wang , Xinzhou Guo , Guoqing Liu , Yang Xiang","doi":"10.1016/j.neunet.2025.107657","DOIUrl":"10.1016/j.neunet.2025.107657","url":null,"abstract":"<div><div>Recently, multimodal electroencephalogram (EEG) learning has shown great promise in disease detection. At the same time, ensuring privacy in clinical studies has become increasingly crucial due to legal and ethical concerns. One widely adopted scheme for privacy protection is differential privacy (DP) because of its clear interpretation and ease of implementation. Although numerous methods have been proposed under DP, it has not been extensively studied for multimodal EEG data due to the complexities of models and signal data considered there. In this paper, we propose a novel Differentially Private Multimodal Laplacian Dropout (DP-MLD) scheme for multimodal EEG learning. Our approach proposes a novel multimodal representative learning model that processes EEG data by language models as text and other modal data by vision transformers as images, incorporating well-designed cross-attention mechanisms to effectively extract and integrate cross-modal features. To achieve DP, we design a novel adaptive feature-level Laplacian dropout scheme, where randomness allocation and performance are dynamically optimized within given privacy budgets. In the experiment on an open-source multimodal dataset of Freezing of Gait (FoG) in Parkinson’s Disease (PD), our proposed method demonstrates an approximate 4% improvement in classification accuracy, and achieves state-of-the-art performance in multimodal EEG learning under DP.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"190 ","pages":"Article 107657"},"PeriodicalIF":6.0,"publicationDate":"2025-06-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144239523","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Progressive discretization for generative retrieval: A self-supervised approach to high-quality DocID generation 渐进式离散化生成检索：一种高质量文档生成的自监督方法

IF 6 1区计算机科学

Neural Networks Pub Date : 2025-06-04 DOI: 10.1016/j.neunet.2025.107663

Shunyu Yao , Jie Hu , Zhiyuan Zhang , Dan Liu

{"title":"Progressive discretization for generative retrieval: A self-supervised approach to high-quality DocID generation","authors":"Shunyu Yao , Jie Hu , Zhiyuan Zhang , Dan Liu","doi":"10.1016/j.neunet.2025.107663","DOIUrl":"10.1016/j.neunet.2025.107663","url":null,"abstract":"<div><div>Generative retrieval is a novel retrieval paradigm where large language models serve as differentiable indices to memorize and retrieve candidate documents in a generative fashion. This paradigm overcomes the limitation that documents and queries must be encoded separately and demonstrates superior performance compared to traditional retrieval methods. To support the retrieval of large-scale corpora, extensive research has been devoted to devising a discrete and distinguishable document representation, namely the DocID. However, most DocIDs are built under unsupervised circumstances, where uncontrollable information distortion will be introduced during the discretization stage. In this work, we propose the <strong>S</strong>elf-supervised <strong>P</strong>rogressive <strong>D</strong>iscretization framework (SPD). SPD first distills document information into multi-perspective continuous representations in a self-supervised way. Then, a progressive discretization algorithm is employed to transform the continuous representations into approximate vectors and discrete DocIDs. The self-supervised model, approximate vectors, and DocIDs are further integrated into a query-side training pipeline to produce an effective generative retriever. Experiments on popular benchmarks demonstrate that SPD builds high-quality search-oriented DocIDs that achieve state-of-the-art generative retrieval performance.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"190 ","pages":"Article 107663"},"PeriodicalIF":6.0,"publicationDate":"2025-06-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144271954","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Consensus multi-view spectral clustering network with unified similarity 具有统一相似度的一致性多视点光谱聚类网络

IF 6 1区计算机科学

Neural Networks Pub Date : 2025-06-04 DOI: 10.1016/j.neunet.2025.107647

Yang Zhao , Daidai Zhu , Aihong Yuan , Xuelong Li

{"title":"Consensus multi-view spectral clustering network with unified similarity","authors":"Yang Zhao , Daidai Zhu , Aihong Yuan , Xuelong Li","doi":"10.1016/j.neunet.2025.107647","DOIUrl":"10.1016/j.neunet.2025.107647","url":null,"abstract":"<div><div>The learning of a consensus representation is critical to multi-view spectral clustering, which generally involves two stages: affinity matrix construction from heterogeneous views and consensus embedding learning for clustering. However, most existing methods construct affinity matrices separately for each view, which hinders the learning of a unified similarity across views. Furthermore, limited attention has been paid to explicitly enforcing consistency among embedding representations, often resulting in suboptimal clustering performance. To address these issues, we propose an effective deep multi-view spectral clustering network to learn consensus representation. Specifically, we first propose a deep spectral embedding learning framework with unified similarity. This framework jointly integrates data to learn a unified similarity across multiple views, and further constructs a spectral mapping network to extract common embedding representations. To further learn sufficient consensus information, we align spectral embedding representations across different views using local structure-constrained contrastive learning. Different samples with high local similarity have their spectral embeddings constrained to be consistent, thus improving the clustering performance. Comparative experiments on eight public datasets validate the superiority and effectiveness of the proposed algorithm.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"190 ","pages":"Article 107647"},"PeriodicalIF":6.0,"publicationDate":"2025-06-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144263667","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Intra-head pruning for vision transformers via inter-layer dimension relationship modeling 基于层间维数关系建模的视觉变形头内剪枝

IF 6 1区计算机科学

Neural Networks Pub Date : 2025-06-04 DOI: 10.1016/j.neunet.2025.107656

Peng Zhang, Cong Tian, Liang Zhao, Zhenhua Duan

{"title":"Intra-head pruning for vision transformers via inter-layer dimension relationship modeling","authors":"Peng Zhang, Cong Tian, Liang Zhao, Zhenhua Duan","doi":"10.1016/j.neunet.2025.107656","DOIUrl":"10.1016/j.neunet.2025.107656","url":null,"abstract":"<div><div>Transformer models have demonstrated good performance across a range of natural language processing and computer vision tasks. However, the huge computational cost imposed by transformer models poses a significant obstacle to their practical implementation on platforms with limited hardware. To address this challenge, recent academic studies have been focused on head pruning, a strategy that effectively eliminates unimportant components in transformer models. Although these pruning methods have shown significant improvements, they suffer from severe accuracy loss due to coarse pruning granularity and fail to consider the interdependence between layers when discarding zero-valued components. This is crucial for achieving a network architecture with efficient compression. Therefore, we propose a novel <strong>i</strong>ntra-<strong>h</strong>ead <strong>p</strong>runing (<strong>IHP</strong>) technique to sparsely train pruned vision transformers. Specifically, our method utilizes a trainable row parameter delicately designed to participate in the sparse training of the model. Furthermore, we introduce a relationship matrix which serves as the key to the grouping pruning process. The grouping policies ensures consistent and coherent elimination of redundant components, thereby maintaining the structural integrity and functional consistency of the pruned network. Experimental results on benchmark datasets (CIFAR-10/100, ImageNet-1K) show that this method can significantly reduce the computational cost of the mainstream vision transformers such as DeiT, Swin Transformer, and CCT, with a small decrease in accuracy. Especially on ILSVRC-12, under the same FLOPs reduction ratio of 46.20%, the Top-1 accuracy improves by 0.47% compared to advanced methods for DeiT-tiny.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"190 ","pages":"Article 107656"},"PeriodicalIF":6.0,"publicationDate":"2025-06-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144263869","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Fast yet versatile machine unlearning for deep neural networks 快速而通用的深度神经网络机器学习

IF 6 1区计算机科学

Neural Networks Pub Date : 2025-06-04 DOI: 10.1016/j.neunet.2025.107648

Kongyang Chen , Dongping Zhang , Bing Mi , Yao Huang , Zhipeng Li

{"title":"Fast yet versatile machine unlearning for deep neural networks","authors":"Kongyang Chen , Dongping Zhang , Bing Mi , Yao Huang , Zhipeng Li","doi":"10.1016/j.neunet.2025.107648","DOIUrl":"10.1016/j.neunet.2025.107648","url":null,"abstract":"<div><div>In response to the growing concerns regarding data privacy, many countries and organizations have implemented corresponding laws and regulations, such as the General Data Protection Regulation (GDPR), to safeguard users’ data privacy. Among these, the <em>Right to Be Forgotten</em> holds particular significance, signifying the necessity for data to be forgotten from improper use. Recently, researchers have integrated the concept of the <em>Right to Be Forgotten</em> into the field of machine learning, focusing on the unlearning of data from machine learning models. However, existing studies either require additional storage for caching updates during the model training phase or are only applicable in specific forgotten scenarios. In this paper, we propose a versatile unlearning method that involves unlearning data by fine-tuning the model until the distribution of the model’s prediction for the forgotten data matches those for unseen third-party data. Importantly, our method does not require additional storage for caching model updates, and it can be applied across different forgotten scenarios. Experimental results demonstrate the efficacy of our method in unlearning backdoor triggers, entire classes of training data, and subsets of training data.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"190 ","pages":"Article 107648"},"PeriodicalIF":6.0,"publicationDate":"2025-06-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144223327","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Supervised contrastive learning with prototype distillation for data incremental learning 基于原型蒸馏的数据增量学习监督对比学习

IF 6 1区计算机科学

Neural Networks Pub Date : 2025-06-03 DOI: 10.1016/j.neunet.2025.107651

Suorong Yang , Tianyue Zhang , Zhiming Xu , Peijia Li , Baile Xu , Furao Shen , Jian Zhao

{"title":"Supervised contrastive learning with prototype distillation for data incremental learning","authors":"Suorong Yang , Tianyue Zhang , Zhiming Xu , Peijia Li , Baile Xu , Furao Shen , Jian Zhao","doi":"10.1016/j.neunet.2025.107651","DOIUrl":"10.1016/j.neunet.2025.107651","url":null,"abstract":"<div><div>The goal of Data Incremental Learning (DIL) is to enable learning from small-scale data batches from non-stationary data streams without clear task divisions. A challenge in this domain is the occurrence of catastrophic forgetting in deep neural networks. To effectively address the challenges inherent to DIL, the trained models must exhibit stability and flexibility, ensuring the retention of information from previously learned classes while adapting to incorporate new ones. Prototypes are particularly effective for classifying separable embeddings within the feature space, as they consolidate embeddings from the same class and push those from different classes further apart. This aligns with the principles of contrastive learning. In this paper, we propose Supervised Contrastive learning with the Prototype Distillation (SCPD) method for the DIL problem. First, we employ supervised contrastive loss (SCL) for model training to enhance the class separability of the extracted model representations and improve the flexibility of the model. To further mitigate the forgetting problem, we propose a prototype distillation loss (PDL), ensuring that feature representations remain close to their corresponding prototypes, enhancing the model’s stability. The integration of SCL and PDL within SCPD ensures both the stability and flexibility of the model. Experimental results demonstrate that the SCPD method outperforms prior state-of-the-art approaches across several benchmarks, including those with various imbalanced setups.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"190 ","pages":"Article 107651"},"PeriodicalIF":6.0,"publicationDate":"2025-06-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144230709","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Feature-decorrelation adaptive contrastive learning for knowledge-aware recommendation 面向知识感知推荐的特征去相关自适应对比学习

IF 6 1区计算机科学

Neural Networks Pub Date : 2025-06-03 DOI: 10.1016/j.neunet.2025.107646

Tong Cai , Yihao Zhang , Kaibei Li , Xiaokang Li , Xibin Wang

{"title":"Feature-decorrelation adaptive contrastive learning for knowledge-aware recommendation","authors":"Tong Cai , Yihao Zhang , Kaibei Li , Xiaokang Li , Xibin Wang","doi":"10.1016/j.neunet.2025.107646","DOIUrl":"10.1016/j.neunet.2025.107646","url":null,"abstract":"<div><div>Knowledge graphs (KGs) are utilized in recommendation systems due to their rich semantic information, with graph neural networks (GNNs) employed to capture multi-hop knowledge and relationships within KGs. However, GNN-based methods, with their iterative linear propagation and the complexity of entity features in KGs, face two significant challenges: (1) The linear iterative aggregation of high-order complex attribute entities can lead to feature loss and distortion in knowledge representation, thereby hindering effective feature modeling; and (2) High-order irrelevant knowledge along the propagation path can cause deviations in recommendation topics. To address these issues, we propose a feature-decorrelation adaptive contrastive learning method for knowledge-aware recommendations. Specifically, we investigate the impact of inter-feature correlations and propose a simple yet effective constraint method to learn representations for downstream tasks. Additionally, we propose an adaptive knowledge refinement method to extract effective high-order semantics from KGs, thereby generating augmented views. Finally, We propose a contrastive learning approach to keep the learned representation focused on the recommended topic and adaptively reduce the negative impact of irrelevant knowledge. We conduct experiments on four public datasets, including Movielens and Yelp, to validate the effectiveness of the proposed method. In particular, our feature decorrelation method demonstrates significant effectiveness in knowledge-aware recommender systems based on GNNs. Our code is available at <span><span>https://github.com/CTimeris/FACLK</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"190 ","pages":"Article 107646"},"PeriodicalIF":6.0,"publicationDate":"2025-06-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144204882","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Incomplete multi-view clustering via efficient anchor tensor recovery framework 基于高效锚张量恢复框架的不完全多视图聚类

IF 6 1区计算机科学

Neural Networks Pub Date : 2025-06-03 DOI: 10.1016/j.neunet.2025.107652

Jintian Ji , Songhe Feng , Jie Huang , Taotao Wei , Xiang Feng , Peiwu Lv , Bing Li

{"title":"Incomplete multi-view clustering via efficient anchor tensor recovery framework","authors":"Jintian Ji , Songhe Feng , Jie Huang , Taotao Wei , Xiang Feng , Peiwu Lv , Bing Li","doi":"10.1016/j.neunet.2025.107652","DOIUrl":"10.1016/j.neunet.2025.107652","url":null,"abstract":"<div><div>Tensor-based Incomplete Multi-view Clustering (TIMC) methods have received widespread attention due to the powerful data recovery capability of capturing cross-view high-order correlation. Although such methods have achieved remarkable progress, they still suffer from the following problems: (1) The extremely high computational complexity makes it hard for tensor-based methods to handle large-scale multi-view data. (2) Geometric structure constraints in the sample space often lead to high computational complexity and redundancy of structural information. (3) The commonly used Tensor Nuclear Norm (TNN) over-penalizes the primary rank components, leading to a sub-optimal representation tensor. Being aware of these, we propose Incomplete Multi-View Clustering with <strong>E</strong>fficient <strong>A</strong>nchor <strong>Te</strong>nsor <strong>R</strong>ecovery Framework (EATER). Specifically, it learns a group of anchors to construct a low-rank anchor tensor to recover the missing data with the high-order correlation among views and the geometric structure in the learned representation tensor is enhanced by an Anchor Laplacian Regularization (ALR). Moreover, instead of employing TNN, we adopt a tighter Non-convex Tensor Rank (NTR) to capture the multi-view high-order correlation effectively. An efficient iterative optimization algorithm is designed to solve the EATER, which is time-economical and enjoys favorable convergence. Extensive experimental results on various datasets demonstrate the superiority of the proposed algorithm as compared to state-of-the-art methods.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"190 ","pages":"Article 107652"},"PeriodicalIF":6.0,"publicationDate":"2025-06-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144211832","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

From ReLU to GeMU: Activation functions in the lens of cone projection 从ReLU到GeMU：锥体投影透镜中的激活函数

IF 6 1区计算机科学

Neural Networks Pub Date : 2025-06-03 DOI: 10.1016/j.neunet.2025.107654

Jiayun Li, Yuxiao Cheng, Yiwen Lu, Zhuofan Xia, Yilin Mo, Gao Huang

引用次数: 0