Neural Networks最新文献

{"title":"Efficient semantic segmentation via logit-guided feature distillation","authors":"Xuyi Yu ,&nbsp;Shang Lou ,&nbsp;Yinghai Zhao ,&nbsp;Huipeng Zhang ,&nbsp;Kuizhi Mei","doi":"10.1016/j.neunet.2026.108663","DOIUrl":"10.1016/j.neunet.2026.108663","url":null,"abstract":"<div><div>Knowledge Distillation (KD) is a critical technique for model compression, facilitating the transfer of implicit knowledge from a teacher model to a more compact, deployable student model. KD can be generally divided into two categories: logit distillation and feature distillation. Feature distillation has been predominant in achieving state-of-the-art (SOTA) performance, but recent advances in logit distillation have begun to narrow the gap. We propose a Logit-guided Feature Distillation (LFD) framework that combines the strengths of both logit and feature distillation to enhance the efficacy of knowledge transfer, particularly leveraging the rich classification information inherent in logits for semantic segmentation tasks. Furthermore, it is observed that Deep Neural Networks (DNNs) only manifest task-relevant characteristics at sufficient depths, which may be a limiting factor in achieving higher accuracy. In this work, we introduce a collaborative distillation method that preemptively focuses on critical pixels and categories in the early stage. We employ logits from deep layers to generate fine-grained spatial masks that are directly conveyed to the feature distillation stage, thereby inducing spatial gradient disparities. Additionally, we generate class masks that dynamically modulate the weights of shallow auxiliary heads, ensuring that class-relevant features can be calibrated by the primary head. A novel shared auxiliary head distillation approach is also presented. Experiments on the Cityscapes, Pascal VOC, and CamVid datasets show that the proposed method achieves competitive performance while maintaining low memory usage. Our codes will be released in <span><span>https://github.com/fate2715/LFD</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"199 ","pages":"Article 108663"},"PeriodicalIF":6.3,"publicationDate":"2026-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146114774","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Resolving ambiguity in code refinement via conidfine: A conversationally-Aware framework with disambiguation and targeted retrieval","authors":"Aoyu Song ,&nbsp;Afizan Azman ,&nbsp;Shanzhi Gu ,&nbsp;Fangjian Jiang ,&nbsp;Jianchi Du ,&nbsp;Tailong Wu ,&nbsp;Mingyang Geng ,&nbsp;Jia Li","doi":"10.1016/j.neunet.2026.108650","DOIUrl":"10.1016/j.neunet.2026.108650","url":null,"abstract":"<div><div>Code refinement is a vital aspect of software development, involving the review and enhancement of code contributions made by developers. A critical challenge in this process arises from unclear or ambiguous review comments, which can hinder developers’ understanding of the required changes. Our preliminary study reveals that conversations between developers and reviewers often contain valuable information that can help resolve such ambiguous review suggestions. However, leveraging conversational data to address this issue poses two key challenges: (1) enabling the model to autonomously determine whether a review suggestion is ambiguous, and (2) effectively extracting the relevant segments from the conversation that can aid in resolving the ambiguity.</div><div>In this paper, we propose a novel method for addressing ambiguous review suggestions by leveraging conversations between reviewers and developers. To tackle the above two challenges, we introduce an <strong>Ambiguous Discriminator</strong> that uses multi-task learning to classify ambiguity and generate type-aware confusion points from a GPT-4-labeled dataset. These confusion points guide a <strong>Type-Driven Multi-Strategy Retrieval Framework</strong> that applies targeted strategies based on categories like <em>Inaccurate Localization, Unclear Expression</em>, and <em>Lack of Specific Guidance</em> to extract actionable information from the conversation context. To support this, we construct a semantic auxiliary instruction library containing spatial indicators, clarification patterns, and action-oriented verbs, enabling precise alignment between review suggestions and informative conversation segments. Our method is evaluated on two widely-used code refinement datasets CodeReview and CodeReview-New, where we demonstrate that our method significantly enhances the performance of various state-of-the-art models, including TransReview, T5-Review, CodeT5, CodeReviewer and ChatGPT. Furthermore, we explore in depth how conversational information improves the model’s ability to address fine-grained situations, and we conduct human evaluations to assess the accuracy of ambiguity detection and the correctness of generated confusion points. We are the first to introduce the issue of ambiguous review suggestions in the code refinement domain and propose a solution that not only addresses these challenges but also sets the foundation for future research. Our method provides valuable insights into improving the clarity and effectiveness of review suggestions, offering a promising direction for advancing code refinement techniques.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"199 ","pages":"Article 108650"},"PeriodicalIF":6.3,"publicationDate":"2026-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146120849","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Warm-start or cold-start? A comparison of generalizability in gradient-based hyperparameter tuning","authors":"Yubo Zhou ,&nbsp;Jun Shu ,&nbsp;Chengli Tan ,&nbsp;Haishan Ye ,&nbsp;Quanziang Wang ,&nbsp;Junmin Liu ,&nbsp;Deyu Meng ,&nbsp;Ivor Tsang ,&nbsp;Guang Dai","doi":"10.1016/j.neunet.2026.108647","DOIUrl":"10.1016/j.neunet.2026.108647","url":null,"abstract":"<div><div>Bilevel optimization (BO) has garnered increasing attention in hyperparameter tuning. BO methods are commonly employed with two distinct strategies for the inner-level: cold-start, which uses a fixed initialization, and warm-start, which uses the last inner approximation solution as the starting point for the inner solver each time, respectively. Previous studies mainly stated that warm-start exhibits better convergence properties, while we provide a detailed comparison of these two strategies from a generalization perspective. Our findings indicate that, compared to the cold-start strategy, warm-start strategy exhibits worse generalization performance, such as more severe overfitting on the validation set. To explain this, we establish generalization bounds for the two strategies. We reveal that warm-start strategy produces a worse generalization upper bound due to its closer interaction with the inner-level dynamics, naturally leading to poor generalization performance. Inspired by the theoretical results, we propose several approaches to enhance the generalization capability of warm-start strategy and narrow its gap with cold-start, especially a novel random perturbation initialization method. Experiments validate the soundness of our theoretical analysis and the effectiveness of the proposed approaches.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"199 ","pages":"Article 108647"},"PeriodicalIF":6.3,"publicationDate":"2026-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146127089","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"SPD-Net: A semantic partitioned transformer with dynamic graph network for improved skeleton-based gait recognition","authors":"Priyanka D,&nbsp;Mala T","doi":"10.1016/j.neunet.2026.108679","DOIUrl":"10.1016/j.neunet.2026.108679","url":null,"abstract":"<div><div>Gait recognition has gained prominence as a biometric modality owing to its unobtrusive and non-invasive nature. Existing methods primarily rely on silhouette-based representations, making them sensitive to variations in clothing, occlusion, and background noise. In contrast, model-based approaches utilize skeleton sequences to capture motion dynamics through joint connectivity, thereby reducing dependence on visual appearance. However, these approaches often rely on physically connected joints, limiting their ability to model semantically meaningful joint relationships. Transformer-based models mitigate this limitation by capturing long-range dependencies, but at the expense of substantial computational overhead. To address these challenges, this work proposes the Semantic Partitioned transformer with Dynamic Graph Network (SPD-Net) for robust gait recognition. SPD-Net integrates Dynamic Graph Convolutional Network (DGCN), Temporal Convolutional Network (TCN), and Semantic Partitioned Multi-head Self-Attention (SP-MSA) to enhance the representation of gait features. DGCN dynamically learns spatial correlations between joints, while TCN captures temporal dependencies. Furthermore, SP-MSA introduces a semantic partitioning strategy that selectively focuses on key joints and frames, significantly reducing computational complexity while preserving crucial gait patterns. This approach effectively models both physically neighboring and distant joint relationships, along with intra- and inter-frame correlations. Finally, a Joint-Part Mapping (JPM) module enhances the discriminative power of gait representations by capturing hierarchical joint relationships across multiple scales. Experimental evaluations on benchmark gait datasets show that SPD-Net surpasses prior state-of-the-art approaches, achieving improved robustness and accuracy across diverse gait recognition challenges.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"199 ","pages":"Article 108679"},"PeriodicalIF":6.3,"publicationDate":"2026-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146174715","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"UniSymNet: A Unified Symbolic Network with Sparse Encoding and Bi-level Optimization","authors":"Xinxin Li ,&nbsp;Juan Zhang ,&nbsp;Da Li ,&nbsp;Xingyu Liu ,&nbsp;Jin Xu ,&nbsp;Junping Yin","doi":"10.1016/j.neunet.2026.108615","DOIUrl":"10.1016/j.neunet.2026.108615","url":null,"abstract":"<div><div>Automatically discovering mathematical expressions is a challenging issue to precisely depict natural phenomena, in which Symbolic Regression (SR) is one of the most widely utilized techniques. Mainstream SR algorithms target on searching for the optimal symbolic tree, but the increasing complexity of the tree structure often limits their performance. Inspired by neural networks, symbolic networks have emerged as a promising new paradigm. However, existing symbolic networks still face certain challenges: binary nonlinear operators { × , ÷} cannot be naturally extended to multivariate, training with fixed architecture often leads to higher complexity and overfitting. In this work, we propose a <strong>Uni</strong>fied <strong>Sym</strong>bolic <strong>Net</strong>work that unifies nonlinear binary operators into nested unary operators, thereby transforming them into multivariate operators. The capability of the proposed UniSymNet is deduced from rigorous theoretical proof, resulting in lower complexity and stronger expressivity. Unlike the conventional neural network training, we design a bi-level optimization framework: the outer level pre-trains a Transformer with sparse label encoding scheme to guide UniSymNet structure selection, while the inner level employs objective-specific strategies to optimize network parameters. This allows for flexible adaptation of UniSymNet structures to different data, leading to reduced expression complexity. The UniSymNet is evaluated on low-dimensional Standard Benchmarks and high-dimensional SRBench, and shows excellent symbolic solution rate, high fitting accuracy, and relatively low expression complexity.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"199 ","pages":"Article 108615"},"PeriodicalIF":6.3,"publicationDate":"2026-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146108216","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Emotion-Aware multimodal deepfake detection","authors":"Teng Zhang ,&nbsp;Gen Li ,&nbsp;Yanhui Xiao ,&nbsp;Huawei Tian ,&nbsp;Yun Cao","doi":"10.1016/j.neunet.2026.108675","DOIUrl":"10.1016/j.neunet.2026.108675","url":null,"abstract":"<div><div>With the continuous advancement of Deepfake techniques, traditional unimodal detection methods struggle to address the challenges posed by multimodal manipulations. Most existing approaches rely on large-scale training data, which limits their generalization to unseen identities or different manipulation types in few-shot settings. In this paper, we propose an emotion-aware multimodal Deepfake detection method that exploits emotion signals for forgery detection. Specifically, we design an emotion embedding extractor (Emoencoder) to capture emotion representations within modalities. Then, we employ Emotion-Aware Contrastive Learning and Cross-Modal Contrastive Learning to capture cross-modal inconsistencies and enhance modality feature extraction. Furthermore, we propose a Text-Guided Semantic Fusion module, where the text modality serves as a semantic anchor to guide audio-visual feature interactions for multimodal feature fusion. To validate our approach under data-limited conditions and unseen identities, we employ a cross-identity few-shot training strategy on benchmark datasets. Experimental results demonstrate that our method outperforms SOTAs and demonstrates superior generalization to both unseen identities and manipulation types.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"199 ","pages":"Article 108675"},"PeriodicalIF":6.3,"publicationDate":"2026-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146133401","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Observer-based prescribed-time optimal neural consensus control for six-rotor UAVs: A novel actor-critic reinforcement learning strategy","authors":"Yue Zhou ,&nbsp;Liang Cao ,&nbsp;Yan Lei ,&nbsp;Hongru Ren","doi":"10.1016/j.neunet.2026.108644","DOIUrl":"10.1016/j.neunet.2026.108644","url":null,"abstract":"<div><div>Six-rotor unmanned aerial vehicles (UAVs) offer significant potential, but still encounter persistent challenges in achieving efficient allocation of limited resources in dynamic and complex environments. Consequently, this paper explores the prescribed-time observer-based optimal consensus control problem for six-rotor UAVs with unified prescribed performance. A practical prescribed-time optimal control scheme is constructed through embedding the prescribed-time control method with a simplified reinforcement learning framework to realize the efficient resource allocation. Leveraging a prescribed-time adjustment function, the novel updating laws for actor and critic neural networks are developed, which guarantee that six-rotor UAVs reach a desired steady state within prescribed time. Moreover, an improved distributed prescribed-time observer is established, ensuring that each follower is able to precisely estimate the velocity and position information of the leader within prescribed time. Then, a series of nonlinear transformations and mappings is proposed, which cannot only satisfy diverse performance requirements under a unified control framework through only adjusting the design parameters a priori but also improve the user-friendliness of implementation and control design. Significantly, the global performance requirement simplifies verification process of initial constraints in traditional performance control methods. Furthermore, an adaptive prescribed-time filter is introduced to address the complexity explosion issue of the backstepping method on six-rotor UAVs, while ensuring the filter error converges within prescribed time. Eventually, simulation results confirm the effectiveness of the designed method.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"199 ","pages":"Article 108644"},"PeriodicalIF":6.3,"publicationDate":"2026-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146138110","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Differentially private data augmentation via LLM generation with discriminative and distribution-aligned filtering","authors":"Yiping Song ,&nbsp;Juhua Zhang ,&nbsp;Zhiliang Tian ,&nbsp;Taishu Sheng ,&nbsp;Yuxin Yang ,&nbsp;Minlie Huang ,&nbsp;Xinwang Liu ,&nbsp;Dongsheng Li","doi":"10.1016/j.neunet.2026.108668","DOIUrl":"10.1016/j.neunet.2026.108668","url":null,"abstract":"<div><div>Data augmentation (DA) is a widely adopted approach for mitigating data insufficiency. Conducting DA in private domains requires privacy-preserving text generation, including anonymization or perturbation applied to sensitive textual data. The above methods lack formal protection guarantees. Existing Differential Privacy (DP) learning methods provide theoretical guarantees by adding calibrated noise to models or outputs. However, the large output space and model scales in text generation require substantial noise, which severely degrades synthesis quality. In this paper, we transfer DP-based synthetic sample generation to DP-based sample discrimination. Specifically, we propose a DP-based DA framework with a large language model (LLM) and a DP-based discriminator for private-domain text generation. Our key idea is to (1) leverage LLMs to generate large-scale high-quality samples, (2) select synthesized samples fitting the private domain, and (3) align the label distribution with the private domain. To achieve this, we use knowledge distillation to construct a DP-based discriminator: teacher models, accessing private data, guide a student model to select samples under calibrated noise. A DP-based tutor further constrains the label distribution of synthesized samples with a low privacy budget. We theoretically analyze the privacy guarantees and empirically validate our method on three medical text classification datasets, showing that our DP-synthesized samples significantly outperform state-of-the-art DP fine-tuning baselines in utility.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"199 ","pages":"Article 108668"},"PeriodicalIF":6.3,"publicationDate":"2026-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146138106","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Lifelong knowledge graph embedding via diffusion model","authors":"Deyu Chen ,&nbsp;Caicai Guo ,&nbsp;Qiyuan Li ,&nbsp;Jinguang Gu ,&nbsp;Meiyi Xie ,&nbsp;Hong Zhu","doi":"10.1016/j.neunet.2026.108630","DOIUrl":"10.1016/j.neunet.2026.108630","url":null,"abstract":"<div><div>Lifelong knowledge graph embedding (KGE) methods aim to learn new knowledge continuously while retaining old knowledge. This line of work has received much attention for its potential to enable knowledge retention and transfer and to reduce training costs under knowledge graphs’ growing scale and flexibility. However, embedding space drift under different contexts is a crucial reason for catastrophic forgetting and inefficient learning of new facts, and existing work ignores this perspective. In order to address the above issues, we proposed a novel lifelong KGE framework that considers learning new facts and preserving old facts in a unified perspective. We propose a diffusion-based embedding method that captures the contextual variation of entity representations and obtains transferable embeddings. In order to handle the drift of the embedding space and balance the learning efficiency, we adopt a reconstruction and generation strategy based on contrastive learning. To avoid catastrophic forgetting and maintain the stability of the embedding distribution, we proposed an effective distribution regularization method. We conduct extensive experiments on seven benchmark datasets with different construction strategies and incremental speed. Experimental results show that our proposed framework outperforms existing lifelong KGE methods.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"199 ","pages":"Article 108630"},"PeriodicalIF":6.3,"publicationDate":"2026-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146081676","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Learnable dendrite neural P systems and applications in survival prediction of glioblastoma patients","authors":"Xiu Yin ,&nbsp;Xiyu Liu ,&nbsp;Shulei Chang ,&nbsp;Bosheng Song ,&nbsp;Guanzhong Gong ,&nbsp;Jiaxing Yin ,&nbsp;Dengwang Li ,&nbsp;Jie Xue","doi":"10.1016/j.neunet.2026.108660","DOIUrl":"10.1016/j.neunet.2026.108660","url":null,"abstract":"<div><div>Current neural-like P systems use “point neurons” as the computing entities, and the computations in these neurons are simplified, ignoring the fact that, in organisms, subcellular compartments (such as neuronal dendrites) can also perform operations as independent computing units in addition to computing at the individual neuron level. The nervous system has a strong ability for optimization learning. Therefore, we propose learnable dendrite neural P (LDNP) systems with new plasticity rules, in which the dendrite structure and learning function can be adaptively changed when solving different application problems. Specifically, the dendrites of neurons are designed as dendritic trees composed of multiple dendritic branches, each of which serves as an independent computing unit. The multilevel complex topological structure of dendrites provides powerful computing capabilities for neurons. A model for predicting the overall survival of glioblastoma (GBM) patients was developed based on LDNP systems and validated on the GBM cohort from the Cancer Genome Atlas. Compared with thirteen state-of-the-art methods, the LDNP system achieves the best performance.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"199 ","pages":"Article 108660"},"PeriodicalIF":6.3,"publicationDate":"2026-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146127098","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}