Neural Networks最新文献_第3页

Towards out-of-distribution detection using gradient vectors 基于梯度向量的分布外检测

IF 6.3 1区计算机科学

Neural Networks Pub Date : 2025-09-25 DOI: 10.1016/j.neunet.2025.108142

Thiago Carvalho , Marley Vellasco , José Franco Amaral

{"title":"Towards out-of-distribution detection using gradient vectors","authors":"Thiago Carvalho , Marley Vellasco , José Franco Amaral","doi":"10.1016/j.neunet.2025.108142","DOIUrl":"10.1016/j.neunet.2025.108142","url":null,"abstract":"<div><div>Deploying Deep Learning algorithms in the real world requires some care that is generally not considered in the training procedure. In real-world scenarios, where the input data cannot be controlled, it is important for a model to identify when a sample does not belong to any known class. This is accomplished using out-of-distribution (OOD) detection, a technique designed to distinguish unknown samples from those that belong to the in-distribution classes. These methods mainly rely on output or intermediate features to calculate OOD scores, but the gradient space is still under-explored for this task. In this work, we propose a new family of methods using gradient features, named GradVec, using the gradient space as input representation for different OOD detection methods. The main idea is that the model gradient presents, in a more informative way, the knowledge that a sample belongs to a known class, being able to distinguish it from other unknown ones. GradVec methods do not change the model training procedure and no additional data is needed to adjust the OOD detector, and it can be used on any pre-trained model. Our approach presents superior results in different scenarios for OOD detection in image classification and text classification, reducing FPR95 up to 26.67 % and 21.29 %, respectively.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"194 ","pages":"Article 108142"},"PeriodicalIF":6.3,"publicationDate":"2025-09-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145221968","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Reinforcement learning with formation energy feedback for material diffusion models 基于地层能量反馈的材料扩散模型强化学习。

IF 6.3 1区计算机科学

Neural Networks Pub Date : 2025-09-24 DOI: 10.1016/j.neunet.2025.108146

Jiao Huang , Qianli Xing , Jinglong Ji , Bo Yang

{"title":"Reinforcement learning with formation energy feedback for material diffusion models","authors":"Jiao Huang , Qianli Xing , Jinglong Ji , Bo Yang","doi":"10.1016/j.neunet.2025.108146","DOIUrl":"10.1016/j.neunet.2025.108146","url":null,"abstract":"<div><div>Generative models are emerging as foundation tools for the discovery of new materials with remarkable efficiency. Existing works introduce physical constraints during the generation process of diffusion models to improve the quality of the generated crystals. However, it is difficult to accurately capture the distribution of stable crystal material structures, given the complex periodic crystal structure and the limited available crystal material data, even with the incorporation of symmetries and other domain-specific knowledge. Thus, these models still struggle to achieve a high success rate in producing stable crystal materials. To further improve the stability of generative crystal materials, we propose a novel fine-tuning framework RLFEF. We formulate the material diffusion process as a Markov Decision Process with formation energy serving as rewards. Moreover, we prove that optimizing the expected return in reinforcement learning is equivalent to applying policy gradient updates to a diffusion model. Additionally, we prove that the fine-tuned model adheres to the unique symmetry of crystal materials. Extensive experiments are conducted on three real-world datasets. The results show that our model achieves state-of-the-art performance on most tasks related to property optimization, ab initio generation, crystal structure prediction, and material generation.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"194 ","pages":"Article 108146"},"PeriodicalIF":6.3,"publicationDate":"2025-09-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145214229","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Graph neural networks for fMRI functional brain networks: A survey. 图神经网络用于fMRI功能脑网络：综述。

IF 6.3 1区计算机科学

Neural Networks Pub Date : 2025-09-24 DOI: 10.1016/j.neunet.2025.108137

Jingye Tang, Tianqing Zhu, Wanlei Zhou, Wei Zhao

{"title":"Graph neural networks for fMRI functional brain networks: A survey.","authors":"Jingye Tang, Tianqing Zhu, Wanlei Zhou, Wei Zhao","doi":"10.1016/j.neunet.2025.108137","DOIUrl":"https://doi.org/10.1016/j.neunet.2025.108137","url":null,"abstract":"<p><p>With the rapid advancement of neuroimaging technologies, the development of deep learning-based models for the analysis of mental disorders has become an emerging consensus. Graphs, as a data and relationship representative, can abstract complex brain data, enabling us to systematically and precisely reveal key issues related to brain structure and function with the support of neuroimaging techniques. Graph neural networks (GNNs) provide new tools and methods for brain network analysis, allowing for a deeper exploration of the relationships between functional regions of the brain and potential functional patterns. Therefore, GNN-based methods for brain network analysis are gaining increasing attention. However, there is currently a lack of a comprehensive summary of the latest research approaches in this field from the perspective of computer science. This survey covers functional brain network analysis methods from different dimensions. In addition, for each method, we discuss the corresponding open challenges and unmet needs to identify the limitations and future directions of these methods in brain network research. Finally, to facilitate researchers in selecting and applying appropriate brain network datasets for experimentation and validation, we summarize the characteristics and sources of various brain network analysis datasets.</p>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"194 ","pages":"108137"},"PeriodicalIF":6.3,"publicationDate":"2025-09-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145259456","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

LLM-led vision-spectral fusion: A zero-shot approach to temporal fruit image classification. llm主导的视觉光谱融合：一种零镜头方法用于水果图像的时间分类。

IF 6.3 1区计算机科学

Neural Networks Pub Date : 2025-09-24 DOI: 10.1016/j.neunet.2025.108155

Huyu Wu, Bowen Jia, Xue-Ming Yuan

引用次数: 0

Implicit graph neural networks with flexible propagation operators 具有柔性传播算子的隐式图神经网络

IF 6.3 1区计算机科学

Neural Networks Pub Date : 2025-09-23 DOI: 10.1016/j.neunet.2025.108143

Yueyang Pi , Yang Huang , Yongquan Shi , Fuhai Chen , Shiping Wang

{"title":"Implicit graph neural networks with flexible propagation operators","authors":"Yueyang Pi , Yang Huang , Yongquan Shi , Fuhai Chen , Shiping Wang","doi":"10.1016/j.neunet.2025.108143","DOIUrl":"10.1016/j.neunet.2025.108143","url":null,"abstract":"<div><div>Due to the capability to capture high-order information of nodes and reduce memory consumption, implicit graph neural networks have become an explored hotspot in recent years. However, these implicit graph neural networks are limited by the static topology, which makes it difficult to handle heterophilic graph-structured data. Furthermore, the existing methods inspired by optimization problem are limited by the explicit structure of graph neural networks, which makes it difficult to set an appropriate number of network layers to solve optimization problems. To address these issues, we propose an implicit graph neural network with flexible propagation operators in this paper. From the optimization objective function, we derive an implicit message passing formula with flexible propagation operators. Compared to the static operator, the proposed method that joints the dynamic semantic and topology of data is more applicable to heterophilic graphs. Moreover, the proposed model performs a fixed-point iterative process for the optimization of the objective function, which implicitly adjusts the number of network layers without requiring sufficient prior knowledge. Extensive experiment results demonstrate the superiority of the proposed model.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"194 ","pages":"Article 108143"},"PeriodicalIF":6.3,"publicationDate":"2025-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145221970","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Adaptive knowledge selection in dialogue systems: Accommodating diverse knowledge types, requirements, and generation models 对话系统中的适应性知识选择：适应不同的知识类型、需求和生成模型。

IF 6.3 1区计算机科学

Neural Networks Pub Date : 2025-09-23 DOI: 10.1016/j.neunet.2025.108133

Yao Zhang , Lang Qin , Zhongtian Bao , Hongru Liang , Jun Wang , Zhenglu Yang , Zhe Sun , Andrzej Cichocki

{"title":"Adaptive knowledge selection in dialogue systems: Accommodating diverse knowledge types, requirements, and generation models","authors":"Yao Zhang , Lang Qin , Zhongtian Bao , Hongru Liang , Jun Wang , Zhenglu Yang , Zhe Sun , Andrzej Cichocki","doi":"10.1016/j.neunet.2025.108133","DOIUrl":"10.1016/j.neunet.2025.108133","url":null,"abstract":"<div><div>Effective knowledge-grounded dialogue systems rely heavily on accurate knowledge selection. This paper begins with an innovative new perspective that categorizes research on knowledge selection based on when knowledge is selected in relation to response generation: pre-, joint-, and post-selection. Among these, pre-selection is of great interest nowadays because they endeavor to provide sufficiently relevant knowledge inputs for downstream response generation models in advance. This reduces the burden of learning, adjusting, and interpreting for the subsequent response generation models, particularly for Large Language Models. Current knowledge pre-selection methods, however, still face three significant challenges: how to cope with different types of knowledge, adapt to the various knowledge requirements in different dialogue contexts, and adapt to different generation models. To resolve the above challenges, we propose ASK, an adaptive knowledge pre-selection method. It unifies various types of knowledge, scores their relevance and contribution to generating desired responses, and adapts the knowledge pool size to ensure the optimal amount is available for generation models. ASK is enhanced by leveraging rewards for selecting appropriate knowledge in both quality and quantity, through a reinforcement learning framework. We perform exhaustive experiments on two benchmarks (WoW and OpenDialKG) and get the following conclusions: 1) ASK has excellent knowledge selection capabilities on diverse knowledge types and requirements. 2) ASK significantly enhances the performance of various downstream generation models, including ChatGPT and GPT-4o. 3) The lightweight improvement of ASK saves 40 % of the computational consumption. Code is available at <span><span>https://github.com/AnonymousCode32213/ASK</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"194 ","pages":"Article 108133"},"PeriodicalIF":6.3,"publicationDate":"2025-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145201974","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

PepHarmony: a multi-view contrastive learning framework for integrated sequence and structure-based peptide representation. PepHarmony：一个多视图对比学习框架，用于整合序列和基于结构的肽表示。

IF 6.3 1区计算机科学

Neural Networks Pub Date : 2025-09-23 DOI: 10.1016/j.neunet.2025.108148

Ruochi Zhang, Haoran Wu, Chang Liu, Huaping Li, Yuqian Wu, Kewei Li, Yifan Wang, Yifan Deng, Jiahui Chen, Fengfeng Zhou, Xin Gao

{"title":"PepHarmony: a multi-view contrastive learning framework for integrated sequence and structure-based peptide representation.","authors":"Ruochi Zhang, Haoran Wu, Chang Liu, Huaping Li, Yuqian Wu, Kewei Li, Yifan Wang, Yifan Deng, Jiahui Chen, Fengfeng Zhou, Xin Gao","doi":"10.1016/j.neunet.2025.108148","DOIUrl":"https://doi.org/10.1016/j.neunet.2025.108148","url":null,"abstract":"<p><p>Recent advances in protein language models have catalyzed significant progress in peptide sequence representation. Despite extensive exploration in this field, pre-trained models tailored for peptide-specific needs remain largely unaddressed due to the difficulty in capturing the complex and sometimes unstable structures of peptides. This study introduces a novel multi-view contrastive learning framework PepHarmony for the sequence-based peptide representation task. PepHarmony innovatively combines sequence- and structure-level information into a sequence-level encoding module through contrastive learning. We carefully select datasets from the Protein Data Bank and AlphaFold DB to encompass a broad spectrum of peptide sequences and structures. The experimental data highlights PepHarmony's exceptional capability in capturing the intricate relationship between peptide sequences and structures compared with the baseline and fine-tuned models. The robustness of our model is confirmed through extensive ablation studies, which emphasize the crucial roles of contrastive loss and strategic data sorting in enhancing predictive performance. The training strategies and the pre-trained PepHarmony model serve as helpful contributions to peptide representations, and offer valuable insights for future applications in peptide drug discovery and peptide engineering.</p>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"194 ","pages":"108148"},"PeriodicalIF":6.3,"publicationDate":"2025-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145245815","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

MSA-LR: Enhancing multi-scale temporal dynamics in multivariate time series forecasting with low-rank self-attention MSA-LR：增强低秩自注意多元时间序列预测的多尺度时间动力学。

IF 6.3 1区计算机科学

Neural Networks Pub Date : 2025-09-23 DOI: 10.1016/j.neunet.2025.108150

Jie Sun , Zhilin Sun , Zhongshan Chen , Mengyang Dong , Xiaozheng Wang , Changwei Chen , Hao Zheng , Xiangjun Zhao

{"title":"MSA-LR: Enhancing multi-scale temporal dynamics in multivariate time series forecasting with low-rank self-attention","authors":"Jie Sun , Zhilin Sun , Zhongshan Chen , Mengyang Dong , Xiaozheng Wang , Changwei Chen , Hao Zheng , Xiangjun Zhao","doi":"10.1016/j.neunet.2025.108150","DOIUrl":"10.1016/j.neunet.2025.108150","url":null,"abstract":"<div><div>Accurately forecasting multivariate time series requires effectively capturing intricate temporal dependencies across diverse scales. Existing deep learning models, while promising, often fall short in this regard. Recurrent architectures like LSTMs struggle with long-range dependencies crucial for multi-scale modeling, while standard Transformers, despite employing attention mechanisms, fail to explicitly differentiate the importance of distinct periodicities, treating all time steps within a fixed window with similar relevance. This limitation hinders their ability to leverage the rich hierarchical structure of real-world time series, particularly in long-term forecasting scenarios. This paper introduces MSA-LR (Multi-Scale Self-Attention with Low-Rank Approximation), a novel architecture explicitly designed to capture multi-scale temporal dynamics. MSA-LR leverages a learnable scale weight matrix and low-rank approximations to directly model the influence of different temporal granularities (e.g., hourly, daily, weekly). This approach not only allows for fine-grained control over multi-scale interactions but also significantly reduces computational complexity compared to standard self-attention, enabling efficient processing of long time series. Empirical evaluations on diverse datasets, including electricity load, traffic flow, and air quality, demonstrate that MSA-LR achieves competitive performance compared to state-of-the-art methods, exhibiting notable improvements in long-term forecasting accuracy. Further analysis reveals MSA-LR's ability to discern and leverage periodic patterns at various resolutions, confirming its effectiveness in capturing the rich multi-scale temporal structure of real-world time series data.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"194 ","pages":"Article 108150"},"PeriodicalIF":6.3,"publicationDate":"2025-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145208155","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Hölder network for improved adversarial robustness Hölder网络，以提高对抗鲁棒性。

IF 6.3 1区计算机科学

Neural Networks Pub Date : 2025-09-23 DOI: 10.1016/j.neunet.2025.108145

Dazhi Zhao , Haiyan Li , Qin Luo , Wenguang Hu

引用次数: 0

Image restoration driven by dual-scale prior 双尺度先验驱动的图像恢复。

IF 6.3 1区计算机科学

Neural Networks Pub Date : 2025-09-23 DOI: 10.1016/j.neunet.2025.108138

Weimin Yuan, Cai Meng, Xiangzhi Bai

{"title":"Image restoration driven by dual-scale prior","authors":"Weimin Yuan, Cai Meng, Xiangzhi Bai","doi":"10.1016/j.neunet.2025.108138","DOIUrl":"10.1016/j.neunet.2025.108138","url":null,"abstract":"<div><div>With the development of advanced imaging technologies, the demand for high quality images in various fields has increased. However, image degradation due to noise, data loss, and other factors persistently hinder image quality. Image restoration (IR) is a critical task in computer vision, aiming to recover original images from degraded observations. Traditional non-learning prior based methods offer flexibility and interpretability but often yield sub-optimal results due to limited representational capacity. In contrast, learning prior based counterparts produce superior performance but suffer from over-fitting and poor generalization to unseen degradations. In this paper, we introduce a novel dual-scale prior (DSP) model that integrates the flexibility strength of non-learning prior with the representation power of learning-based prior. Specifically, the DSP model employs a group-scale physical prior, leveraging non-local self-similarity (NSS) for jointly sparse and low-rank approximation. And an image-scale bias-free deep denoising prior for capturing external characteristics. These dual-scale priors complement each other by effectively preserving edges and removing noise, demonstrating robustness across various types of degradation. We then present DSPIR, an effective IR method by incorporating DSP into existing maximum a posteriori (MAP) principle. DSPIR is solved by alternating minimization and alternating direction method of multipliers. Extensive evaluations on both synthetic and real data demonstrate that DSPIR achieves better performance in image denoising and inpainting compared to state-of-the-art methods.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"194 ","pages":"Article 108138"},"PeriodicalIF":6.3,"publicationDate":"2025-09-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145193715","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0