Neural Networks最新文献

{"title":"Multi-modal sentiment recognition with residual gating network and emotion intensity attention","authors":"Yadi Wang ,&nbsp;Xiaoding Guo ,&nbsp;Xianhong Hou ,&nbsp;Zhijun Miao ,&nbsp;Xiaojin Yang ,&nbsp;Jinkai Guo","doi":"10.1016/j.neunet.2025.107483","DOIUrl":"10.1016/j.neunet.2025.107483","url":null,"abstract":"<div><div>Multimodal emotion recognition focuses on the prediction of emotions using text, visual and acoustic modalities, and some results have been generated in this field. Previous approaches fall short in two aspects, one is the processing of complementary information among modalities, the other is how to avoid the long-term dependency and select the most important joint modal features. In this paper, we propose a new multimodal emotion recognition framework MSRG, which consists of feature extraction (FE), emotional intensity attention (EIA), time-step level fusion (TLF), utterance level fusion (ULF), and sentiment inference module (SIM). EIA is divided into adaptive multimodal linear pooling (AMLP) and joint cross-attention fusion (JCAF), where AMLP adopts the adaptive strategy of multimodal fusion to dynamically calculate the adaptive coefficients of three modalities, then performs the pooling operation to obtain joint modal features. JCAF calculates the attention weights and attention features of each modality based on cross-correlation between individual and joint features. TLF performs feature alignment fusion at the time-step level, then uses the residual gating network (RGN) to process the time-step level fused sequences. The obtained time-step level fused features are then input into two fully connected layers and an activation layer to obtain the time-step level emotion intensity. ULF fuses the three modalities’ utterance level representations by concatenating them and then inputs the obtained utterance level fused features into a fully connected layer to obtain the utterance level emotion intensity. Finally, both the time-step level emotion intensity and the utterance level emotion intensity are input into SIM to obtain the final emotion prediction results. Experiments demonstrate that MSRG achieves better prediction performance on CMU-MOSI and CMU-MOSEI datasets.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"188 ","pages":"Article 107483"},"PeriodicalIF":6.0,"publicationDate":"2025-04-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143895237","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Communication-efficient distributed learning with Local Immediate Error Compensation","authors":"Yifei Cheng ,&nbsp;Li Shen ,&nbsp;Linli Xu ,&nbsp;Xun Qian ,&nbsp;Shiwei Wu ,&nbsp;Yiming Zhou ,&nbsp;Tie Zhang ,&nbsp;Dacheng Tao ,&nbsp;Enhong Chen","doi":"10.1016/j.neunet.2025.107471","DOIUrl":"10.1016/j.neunet.2025.107471","url":null,"abstract":"<div><div>Gradient compression with error compensation has attracted significant attention with the target of reducing the heavy communication overhead in distributed learning. However, existing compression methods either perform only unidirectional compression in one iteration with higher communication cost, or bidirectional compression with slower convergence rate. In this work, we propose the Local Immediate Error Compensated SGD (LIEC-SGD) optimization algorithm to break the above bottlenecks based on bidirectional compression and carefully designed compensation approaches. Specifically, the bidirectional compression technique is to reduce the communication cost, and the compensation technique compensates the local compression error to the model update immediately while only maintaining the global error variable on the server throughout the iterations to boost its efficacy. Theoretically, we prove that LIEC-SGD is superior to previous works in either the convergence rate or the communication cost, which indicates that LIEC-SGD could inherit the dual advantages from unidirectional compression and bidirectional compression. Finally, experiments of training deep neural networks validate the effectiveness of the proposed LIEC-SGD algorithm. When adopting two compression operators, the best test accuracies of LIEC-SGD are higher than the second best baseline with 0.53% and 0.33% on CIFAR-10, 1.39% and 1.44% on CIFAR-100. From the wall-clock time perspective, LIEC-SGD respectively achieves <span><math><mrow><mn>1</mn><mo>.</mo><mn>428</mn><mo>×</mo></mrow></math></span> and <span><math><mrow><mn>1</mn><mo>.</mo><mn>721</mn><mo>×</mo></mrow></math></span> speedup over parallel SGD on two CIFAR datasets.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"188 ","pages":"Article 107471"},"PeriodicalIF":6.0,"publicationDate":"2025-04-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143880916","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Kernel-free quadratic surface SVM for conditional probability estimation in imbalanced multi-class classification","authors":"Junyou Ye,&nbsp;Zhixia Yang,&nbsp;Yongqi Zhu,&nbsp;Zheng Zhang,&nbsp;Qin Wen","doi":"10.1016/j.neunet.2025.107480","DOIUrl":"10.1016/j.neunet.2025.107480","url":null,"abstract":"<div><div>For the multi-class classification problems, we propose a new probabilistic output classifier called kernel-free quadratic surface support vector machine for conditional probability estimation (CPSQSVM), which is based on a newly developed binary classifier (BCPSQSVM) combined with the one vs. rest (OvR) decomposition strategy. The purpose of BCPSQSVM is to estimate the positive class posterior conditional probability density and assume it to be a quadratic function. Further, the definition of quadratically separable in probability is given and the optimization problem of BCPSQSVM is constructed under its guidance. The primal problem can be solved directly, because it is a convex quadratic programming problem (QPP) without using kernel functions. However, we design the corresponding block iteration algorithm for its dual problem, which perhaps rendered the device inoperable due to the large constraint size of the primal problem. It is worth noting that our CPSQSVM assigns greater weights to minority samples to mitigate the negative impact of labeling imbalance due to the use of OvR strategy. The existence and uniqueness of optimal solutions, as well as the reliability and versatility of CPSQSVM are discussed in the theoretical analysis. In addition, convergence of the algorithm and upper bound on the margin parameter are analyzed. The feasibility and validity of the proposed method is verified by numerical experiments on some artificial and benchmark datasets.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"188 ","pages":"Article 107480"},"PeriodicalIF":6.0,"publicationDate":"2025-04-25","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143895860","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Multi-Granularity Autoformer for long-term deterministic and probabilistic power load forecasting","authors":"Yang Yang ,&nbsp;Yuchao Gao ,&nbsp;Hu Zhou ,&nbsp;Jinran Wu ,&nbsp;Shangce Gao ,&nbsp;You-Gan Wang","doi":"10.1016/j.neunet.2025.107493","DOIUrl":"10.1016/j.neunet.2025.107493","url":null,"abstract":"<div><div>Long-term power load forecasting is critical for power system planning but is constrained by intricate temporal patterns. Transformer-based models emphasize modeling long- and short-term dependencies yet encounter limitations from complexity and parameter overhead. This paper introduces a novel Multi-Granularity Autoformer (MG-Autoformer) for long-term load forecasting. The model leverages a Multi-Granularity Auto-Correlation Attention Mechanism (MG-ACAM) to effectively capture fine-grained and coarse-grained temporal dependencies, enabling accurate modeling of short-term fluctuations and long-term trends. To enhance efficiency, a shared query–key (Q–K) mechanism is utilized to identify key temporal patterns across multiple resolutions and reduce model complexity. To address uncertainty in power load forecasting, the model incorporates a quantile loss function, enabling probabilistic predictions while quantifying uncertainty. Extensive experiments on benchmark datasets from Portugal, Australia, America, and ISO New England demonstrate the superior performance of the proposed MG-Autoformer in long-term power load point and probabilistic forecasting tasks.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"188 ","pages":"Article 107493"},"PeriodicalIF":6.0,"publicationDate":"2025-04-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143873743","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Pair-wise or high-order? A self-adaptive graph framework for knowledge graph embedding","authors":"Dong Zhang ,&nbsp;Haoqian Jiang ,&nbsp;Xiaoning Li ,&nbsp;Guanyu Li ,&nbsp;Bo Ning ,&nbsp;Heng Chen","doi":"10.1016/j.neunet.2025.107494","DOIUrl":"10.1016/j.neunet.2025.107494","url":null,"abstract":"<div><div>Knowledge graphs (KGs) depict entities as nodes and connections as edges, and they are extensively utilized in numerous artificial intelligence applications. However, knowledge graphs often suffer from incompleteness, which seriously affects downstream applications. Knowledge graph embedding (KGE) technology tackles this challenge by encoding entities and relations as vectors, allowing for inference and computations using these representations. Graph convolutional networks (GCNs) are essential knowledge graph embedding technology models. GCN methods capture the topological structure features of the KG by calculating the pair-wise relationships between entities and neighboring nodes. Although GCN models have achieved excellent performance, there are still three main challenges: (1) effectively solving the over-smoothing problem in multi-layer GCN models for knowledge graph representation learning, (2) obtaining high-order information between entities and neighboring nodes beyond the pair-wise relationships, and (3) effectively integrating pair-wise and high-order features of entities. To address these challenges, we proposed an adaptive graph convolutional network model called PHGCN (Pair-wise and High-Order Graph Convolutional Network), which can simultaneously integrate pair-wise and high-order features. PHGCN tackles the three challenges mentioned above in the following ways. (1) We propose a layer-aware GCN to overcome the over-smoothing problem while aggregating pair-wise relationships. (2) We employ simplicial complex neural networks to extract high-order topological features from knowledge graphs. (3) We introduce a self-adaptive aggregation mechanism that effectively integrates pair-wise and high-order features. Our experiments on four benchmark datasets showed that PHGCN outperforms existing methods, achieving state-of-the-art results. The performance improvement from using a simplicial complex neural network to extract high-order features is significant. On the FB15k-237 dataset, PHGCN achieved a 1.5% improvement, while on the WN18RR dataset, it improved by 6.1%.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"188 ","pages":"Article 107494"},"PeriodicalIF":6.0,"publicationDate":"2025-04-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143873744","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Linking cellular-level phenomena to brain architecture: the case of spiking cerebellar controllers","authors":"Egidio D’Angelo ,&nbsp;Alberto Antonietti ,&nbsp;Alice Geminiani ,&nbsp;Benedetta Gambosi ,&nbsp;Cristiano Alessandro ,&nbsp;Emiliano Buttarazzi ,&nbsp;Alessandra Pedrocchi ,&nbsp;Claudia Casellato","doi":"10.1016/j.neunet.2025.107538","DOIUrl":"10.1016/j.neunet.2025.107538","url":null,"abstract":"<div><div>Linking cellular-level phenomena to brain architecture and behavior is a holy grail for theoretical and computational neuroscience. Advances in neuroinformatics have recently allowed scientists to embed spiking neural networks of the cerebellum with realistic neuron models and multiple synaptic plasticity rules into sensorimotor controllers. By minimizing the distance (error) between the desired and the actual sensory state, and exploiting the sensory prediction, the cerebellar network acquires knowledge about the body-environment interaction and generates corrective signals. In doing so, the cerebellum implements a generalized computational algorithm, allowing it \"<em>to learn to predict the timing between correlated events</em>\" in a rich set of behavioral contexts. Plastic changes evolve trial by trial and are distributed over multiple synapses, regulating the timing of neuronal discharge and fine-tuning high-speed movements on the millisecond timescale. Thus, spiking cerebellar built-in controllers, among various computational approaches to studying cerebellar function, are helping to reveal the cellular-level substrates of network learning and signal coding, opening new frontiers for predictive computing and autonomous learning in robots.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"188 ","pages":"Article 107538"},"PeriodicalIF":6.0,"publicationDate":"2025-04-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143923378","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Integrated codec decomposed Transformer for long-term series forecasting","authors":"Benhan Li ,&nbsp;Wei Zhang ,&nbsp;Mingxin Lu","doi":"10.1016/j.neunet.2025.107484","DOIUrl":"10.1016/j.neunet.2025.107484","url":null,"abstract":"<div><div>Recently, Transformer-based and multilayer perceptron (MLP) based architectures have formed a competitive landscape in the field of time series forecasting. There is evidence that series decomposition can further enhance the model’s ability to perceive temporal patterns. However, most of the existing Transformer-based decomposed models capture seasonal features progressively and assist in adding trends for forecasting, but ignore the deep information contained in trends and may lead to pattern mismatch in the fusion stage. In addition, the permutation invariance of the attention mechanism inevitably leads to the loss of temporal order. After in-depth analysis of the applicability of attention and linear layers to series components, we propose to use attention to learn multivariate correlations from trends, and MLP to capture seasonal patterns. We further introduce an integrated codec that provides the same multivariate relationship representation for both the encoding and decoding stages, ensuring effective inheritance of temporal dependencies. To mitigate the fading of sequentiality during attention, we propose trend enhancement module, which maintains the stability of the trend by expanding the series to a longer time scale, helping the attention mechanism to achieve fine-grained feature representations. Extensive experiments show that our model exhibits state-of-the-art prediction performance on large-scale datasets.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"188 ","pages":"Article 107484"},"PeriodicalIF":6.0,"publicationDate":"2025-04-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143886453","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Red alarm: Controllable backdoor attack in continual learning","authors":"Rui Gao,&nbsp;Weiwei Liu","doi":"10.1016/j.neunet.2025.107479","DOIUrl":"10.1016/j.neunet.2025.107479","url":null,"abstract":"<div><div>Continual learning (CL) studies the problem of learning a single model from a sequence of disjoint tasks. The main challenge is to learn without catastrophic forgetting, a scenario in which the model’s performance on previous tasks degrades significantly as new tasks are added. However, few works focus on the security challenge in the CL setting. In this paper, we focus on the backdoor attack in the CL setting. Specifically, we provide the threat model and explore what attackers in a CL setting will face. Based on these findings, we propose a controllable backdoor attack mechanism in continual learning (CBACL). Experimental results on the Split Cifar and Tiny Imagenet datasets confirm the advantages of our proposed mechanism.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"188 ","pages":"Article 107479"},"PeriodicalIF":6.0,"publicationDate":"2025-04-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143877169","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Saccade and purify: Task adapted multi-view feature calibration network for few shot learning","authors":"Jing Zhang,&nbsp;Yunzuo Hu,&nbsp;Xinzhou Zhang,&nbsp;Mingzhe Chen,&nbsp;Zhe Wang","doi":"10.1016/j.neunet.2025.107482","DOIUrl":"10.1016/j.neunet.2025.107482","url":null,"abstract":"<div><div>Current few-shot image classification methods encounter challenges in extracting multi-view features that can complement each other and selecting optimal features for classification in a specific task. To address this problem, we propose a novel Task-adapted Multi-view feature Calibration Network (TMCN) inspired by the different saccade patterns observed in the human visual system. The TMCN is designed to “saccade” for extracting complementary multi-view features and “purify” multi-view features in a task-adapted manner. To capture more representative features, we propose a multi-view feature extraction method that simulates the voluntary saccades and scanning saccades in the human visual system, which generates global, local grid, and randomly sampled multi-view features. To purify and obtain the most appropriate features, we employ a global local feature calibration module to calibrate global and local grid features for achieving more stable non-local image features. Furthermore, a sampling feature fusion method is proposed to fuse the randomly sampled features from classes to obtain better prototypes, and a multi-view feature calibrating module is proposed to adaptively fuse purified multi-view features based on the task information obtained from the task feature extracting module. Extensive experiments conducted on three widely used public datasets prove that our proposed TMCN can achieve excellent performance and surpass state-of-the-art methods. The code is available at the following address: <span><span>https://github.com/huyunzuo/TMCN</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"188 ","pages":"Article 107482"},"PeriodicalIF":6.0,"publicationDate":"2025-04-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143880919","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Enhancing motor imagery EEG classification with a Riemannian geometry-based spatial filtering (RSF) method","authors":"Lincong Pan ,&nbsp;Kun Wang ,&nbsp;Yongzhi Huang ,&nbsp;Xinwei Sun ,&nbsp;Jiayuan Meng ,&nbsp;Weibo Yi ,&nbsp;Minpeng Xu ,&nbsp;Tzyy-Ping Jung ,&nbsp;Dong Ming","doi":"10.1016/j.neunet.2025.107511","DOIUrl":"10.1016/j.neunet.2025.107511","url":null,"abstract":"<div><div>Motor imagery (MI) refers to the mental simulation of movements without physical execution, and it can be captured using electroencephalography (EEG). This area has garnered significant research interest due to its substantial potential in brain-computer interface (BCI) applications, especially for individuals with physical disabilities. However, accurate classification of MI EEG signals remains a major challenge due to their non-stationary nature, low signal-to-noise ratio, and sensitivity to both external and physiological noise. Traditional classification methods, such as common spatial pattern (CSP), often assume that the data is stationary and Gaussian, which limits their applicability in real-world scenarios where these assumptions do not hold. These challenges highlight the need for more robust methods to improve classification accuracy in MI-BCI systems. To address these issues, this study introduces a Riemannian geometry-based spatial filtering (RSF) method that projects EEG signals into a lower-dimensional subspace, maximizing the Riemannian distance between covariance matrices from different classes. By leveraging the inherent geometric properties of EEG data, RSF enhances the discriminative power of the features while maintaining robustness against noise. The performance of RSF was evaluated in combination with ten commonly used MI decoding algorithms, including CSP with linear discriminant analysis (CSP-LDA), Filter Bank CSP (FBCSP), Minimum Distance to Riemannian Mean (MDM), Tangent Space Mapping (TSM), EEGNet, ShallowConvNet (sCNN), DeepConvNet (dCNN), FBCNet, Graph-CSPNet, and LMDA-Net, using six publicly available MI-BCI datasets. The results demonstrate that RSF significantly improves classification accuracy and reduces computational time, particularly for deep learning models with high computational complexity. These findings underscore the potential of RSF as an effective spatial filtering approach for MI EEG classification, providing new insights and opportunities for the development of robust MI-BCI systems. The code for this research is available at <span><span>https://github.com/PLC-TJU/RSF</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"188 ","pages":"Article 107511"},"PeriodicalIF":6.0,"publicationDate":"2025-04-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143877170","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}