Neural Networks最新文献

{"title":"Learning performance of cerebellar circuit depends on diversity and chaoticity of spiking patterns in granule cells: A simulation study","authors":"Soichiro Fujiki,&nbsp;Kenji Kansaku","doi":"10.1016/j.neunet.2025.107585","DOIUrl":"10.1016/j.neunet.2025.107585","url":null,"abstract":"<div><div>The cerebellum, composed of numerous neurons, plays various roles in motor control. Although it is functionally subdivided, the cerebellar cortex has a canonical structural pattern in neuronal circuits including a recurrent circuit pattern formed by granule cells (GrCs) and Golgi cells (GoCs). The canonical circuital pattern suggests the existence of a fundamental computational algorithm, although it remains unclear. Modeling and simulation studies are useful for verifying hypotheses about complex systems. Previous models have shown that they could reproduced the neurophysiological data of the cerebellum; however, the dynamic characteristics of the system have not been fully elucidated. Understanding the dynamic characteristics of the circuital pattern is necessary to reveal the computational algorithm embedded in the circuit. This study conducted numerical simulations using the cerebellar circuit model to investigate dynamic characteristics in a simplified model of cerebellar microcircuits. First, the diversity and chaoticity of the patterns of spike trains generated from GrCs depending on the synaptic strength between the GrCs and GoCs were investigated based on cluster analysis and the Lyapunov exponent, respectively. Then the effect of synaptic strength on learning tasks was investigated based on the convergence properties of the output signals from Purkinje cells. The synaptic strength for high learning performance was almost consistent with that for the high diversity of the generated patterns and the edge of chaos. These results suggest that the learning performance of the cerebellar circuit depends on the diversity and the chaoticity of the spiking patterns from the GrC–GoC recurrent circuit.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"189 ","pages":"Article 107585"},"PeriodicalIF":6.0,"publicationDate":"2025-05-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143934686","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Interpretable inverse iteration mean shift networks for clustering tasks","authors":"Bingjie Zhang ,&nbsp;Zihan Yu ,&nbsp;Jian Wang ,&nbsp;Maciej Żelaszczyk ,&nbsp;Yaqian Zhang ,&nbsp;Jacek Mańdziuk ,&nbsp;Chao Zhang ,&nbsp;Witold Pedrycz","doi":"10.1016/j.neunet.2025.107528","DOIUrl":"10.1016/j.neunet.2025.107528","url":null,"abstract":"<div><div>Neural networks have become the standard approach for tasks such as computer vision, machine translation and pattern recognition. While they exhibit significant feature representation capabilities, they often lack interpretability. This suggests that it might be beneficial to explore interpretable machine learning approaches in order to inform the neural network architectures. On the contrary, the mean shift algorithm (MS) has an unambiguous computational process, yet lacking in representation power. In order to draw on the advantages of both neural networks and the mean shift method, we propose the mean shift network (MS-Net) — a novel architecture, which is an inverse iteration fuzzy clustering network. Each layer of the proposed network possesses good interpretability, while the network as a whole is able to produce strong feature representations. To relax the limitations of the kernel function and ensure convergence, we design a continuously-differentiable Gaussian-inspired kernel as the activation function for the membership layer of MS-Net. Furthermore, we devise a weighted version of the architecture, called WMS-Net, to incorporate the importance of the training examples. We present theoretical results, with proofs of weak and strong convergence. We also consider two extensions to the proposed method, CB-MS-Net and CB-WMS-Net, which apply the curvature-based method to MS-Net and WMS-Net. Simulation results on 5 clustering tasks and 6 real-world datasets confirm the effectiveness of the proposed algorithms.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"189 ","pages":"Article 107528"},"PeriodicalIF":6.0,"publicationDate":"2025-05-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143929460","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Entity replacement strategy for temporal knowledge graph query relaxation","authors":"Luyi Bai ,&nbsp;Jixuan Dong ,&nbsp;Lin Zhu","doi":"10.1016/j.neunet.2025.107579","DOIUrl":"10.1016/j.neunet.2025.107579","url":null,"abstract":"<div><div>The temporal knowledge graph (TKG) query enables the retrieval of candidate answer lists by addressing questions that involve temporal constraints, regarded as a crucial downstream task in the realm of the temporal knowledge graph. Existing methods primarily focus on the TKG queries of non-empty results, while neglecting the consideration of TKG queries that return empty results. Therefore, there is still potential for enhancing the flexibility of queries. In this paper, we propose an <strong>E</strong>ntity <strong>R</strong>eplacement strategy for <strong>T</strong>emporal knowledge graph <strong>Q</strong>uery <strong>R</strong>elaxation (ER-TQR), a flexible relaxation method for TKG queries targeting empty results based on an entity replacement strategy. ER-TQR distinguishes itself from existing query relaxation techniques replacing incompatible entities with semantically and temporally aligned candidates, minimizing distortion of original queries. For the query embedding, we leverage an embedding method based on the <strong>B</strong>idirectional <strong>E</strong>ncoder <strong>R</strong>epresentations from <strong>T</strong>ransformers (BERT) model, which significantly improves the semantic representation ability. Concurrently, we use the <strong>B</strong>idirectional <strong>G</strong>ated <strong>R</strong>ecurrent <strong>U</strong>nit (Bi-GRU) model to assess the chance of each entity appearing with errors and decide if it needs to be replaced. To uphold the original intent of the query, we replace the entities based on similarity calculation and generate relaxed query results. The experimental results show that our method outperforms existing query relaxation methods in 4 out of 5 metrics on different datasets.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"188 ","pages":"Article 107579"},"PeriodicalIF":6.0,"publicationDate":"2025-05-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143927657","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"CLPNets: Coupled Lie–Poisson neural networks for multi-part Hamiltonian systems with symmetries","authors":"Christopher Eldred ,&nbsp;François Gay-Balmaz ,&nbsp;Vakhtang Putkaradze","doi":"10.1016/j.neunet.2025.107441","DOIUrl":"10.1016/j.neunet.2025.107441","url":null,"abstract":"<div><div>To accurately compute data-based prediction of Hamiltonian systems, it is essential to utilize methods that preserve the structure of the equations over time. We consider a particularly challenging case of systems with interacting parts that do not reduce to pure momentum evolution. Such systems are essential in scientific computations, such as discretization of a continuum elastic rod, which can be viewed as the group of rotations and translations <span><math><mrow><mi>S</mi><mi>E</mi><mrow><mo>(</mo><mn>3</mn><mo>)</mo></mrow></mrow></math></span>. The evolution involves not only the momenta but also the relative positions and orientations of the particles. The presence of Lie group-valued elements, such as relative positions and orientations, poses a problem for applying previously derived methods for data-based computing. We develop a novel method of data-based computation and complete phase space learning of such systems. We follow the original framework of <em>SympNets</em> (Jin et al., 2020) and <em>LPNets</em> (Eldred et al., 2024), building the neural network from phase space mappings that preserve the Lie–Poisson structure. We derive a novel system of mappings that are built into neural networks describing the evolution of such systems. We call such networks Coupled Lie–Poisson Neural Networks, or <em>CLPNets</em>. We consider increasingly complex examples for the applications of CLPNets, starting with the rotation of two rigid bodies about a common axis, progressing to the free rotation of two rigid bodies, and finally to the evolution of two connected and interacting <span><math><mrow><mi>S</mi><mi>E</mi><mrow><mo>(</mo><mn>3</mn><mo>)</mo></mrow></mrow></math></span> components, describing the discretization of an elastic rod into two elements. Our method preserves all Casimir invariants to machine precision, preserves energy to high accuracy, and shows good resistance to the curse of dimensionality, requiring only a few thousand data points for all cases studied (three to eighteen dimensions). Additionally, the method is highly economical in memory requirements, requiring only about 200 parameters for the most complex case considered.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"189 ","pages":"Article 107441"},"PeriodicalIF":6.0,"publicationDate":"2025-05-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143916910","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Data alignment based adversarial defense benchmark for EEG-based BCIs","authors":"Xiaoqing Chen ,&nbsp;Tianwang Jia ,&nbsp;Dongrui Wu","doi":"10.1016/j.neunet.2025.107516","DOIUrl":"10.1016/j.neunet.2025.107516","url":null,"abstract":"<div><div>Machine learning has been extensively applied to signal decoding in electroencephalogram (EEG)-based brain–computer interfaces (BCIs). While most studies have focused on enhancing the accuracy of EEG-based BCIs, more attention should be given to their security. Recent findings reveal that EEG-based BCIs are vulnerable to adversarial attacks. To address this, we present the first adversarial defense benchmark based on data alignment, aiming to enhance both the accuracy and robustness of EEG-based BCIs. This study evaluates nine adversarial defense approaches (including five defense strategies) across five EEG datasets (covering three paradigms), three neural networks, and four experimental scenarios. Our results show for the first time that integrating data augmentation, data alignment, and robust training can further improve both the accuracy and robustness of BCIs compared to using only one or two of them. Furthermore, we provide insights into the characteristics of various adversarial defense approaches based on EEG data alignment, offering valuable guidance for developing more accurate and secure EEG-based BCIs.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"188 ","pages":"Article 107516"},"PeriodicalIF":6.0,"publicationDate":"2025-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143906387","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Federated learning with randomized alternating direction method of multipliers and application in training neural networks","authors":"Duy Nhat Phan ,&nbsp;Patrick Hytla ,&nbsp;Andrew Rice ,&nbsp;Thuy Ngoc Nguyen","doi":"10.1016/j.neunet.2025.107501","DOIUrl":"10.1016/j.neunet.2025.107501","url":null,"abstract":"<div><div>Federated learning (FL) is a research area focusing on model training across numerous users while preserving data privacy under the coordination of a central server. The inherent optimization challenges in FL often manifest as nonconvex and nonsmooth problems, presenting significant computational difficulties. This paper proposes a novel FL algorithm that combines the alternating direction method of multipliers (<span>ADMM</span>) with a randomized block-coordinate strategy and general majorization-minimization principle. We provide almost surely subsequential convergence of the generated sequence to a stationary point. We show that our algorithm possesses the best-known complexity bound in terms of the number of communication rounds. Further, through empirical evaluations on well-known datasets, we demonstrate the effectiveness of our algorithm on classification problems using neural networks, underscoring its practical efficacy in real-world applications.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"189 ","pages":"Article 107501"},"PeriodicalIF":6.0,"publicationDate":"2025-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143928170","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Neuronal correlates of sleep in honey bees","authors":"Sebastian Moguilner ,&nbsp;Ettore Tiraboschi ,&nbsp;Giacomo Fantoni ,&nbsp;Heather Strelevitz ,&nbsp;Hamid Soleimani ,&nbsp;Luca Del Torre ,&nbsp;Uri Hasson ,&nbsp;Albrecht Haase","doi":"10.1016/j.neunet.2025.107575","DOIUrl":"10.1016/j.neunet.2025.107575","url":null,"abstract":"<div><div>Honey bees <em>Apis mellifera</em> follow the day-night cycle for their foraging activity, entering rest periods during darkness. Despite considerable research on sleep behaviour in bees, its underlying neurophysiological mechanisms are not well understood, partly due to the lack of brain imaging data that allow for analysis from a network- or system-level perspective.</div><div>This study aims to fill this gap by investigating whether neuronal activity during rest periods exhibits stereotypic patterns comparable to sleep signatures observed in vertebrates. Using two-photon calcium imaging of the antennal lobes (AL) in head-fixed bees, we analysed brain dynamics across motion and rest epochs during the nocturnal period. The recorded activity was computationally characterised, and machine learning was applied to determine whether a classifier could distinguish the two states after motion correction. Out-of-sample classification accuracy reached 93 %, and a feature importance analysis suggested network features to be decisive. Accordingly, the glomerular connectivity was found to be significantly increased in the rest-state patterns. A full simulation of the AL using a leaky spiking neural network revealed that such a transition in network connectivity could be achieved by weakly correlated input noise and a reduction of synaptic conductance of the inhibitive local neurons (LNs) which couple the AL network nodes. The difference in the AL response maps between awake- and sleep-like states generated by the simulation showed a decreased specificity of the odour code in the sleep state, suggesting reduced information processing during sleep. Since LNs in the bee brain are GABAergic, this suggests that the GABAergic system plays a central role in sleep regulation in bees as in many higher species including humans. Our findings support the theoretical view that sleep-related network modulation mechanisms are conserved throughout evolution, highlighting the bee’s potential as an invertebrate model for studying sleep at the level of single neurons.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"189 ","pages":"Article 107575"},"PeriodicalIF":6.0,"publicationDate":"2025-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143931335","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Ex2Vec: Enhancing assembly code semantics with end-to-end execution-aware embeddings","authors":"Xingyu Gong ,&nbsp;Yang Xu ,&nbsp;Sicong Zhang ,&nbsp;Chenhang He","doi":"10.1016/j.neunet.2025.107506","DOIUrl":"10.1016/j.neunet.2025.107506","url":null,"abstract":"<div><div>Binary code similarity detection (BSCD), whose goal is to identify and analyze similar or identical functions in compiled binaries, is an essential task in computer security. Recent methods leveraging deep neural networks (DNN) for numerical vector representation of code have achieved significant success. However, these methods primarily adapt techniques from masked language modeling (MLM), encoding code instructions by predicting missing values from an instruction context, which limits their ability to fully capture execution semantics. In this paper, we propose Ex2vec, an innovative end-to-end encoding method that generates high-quality embeddings rich in execution semantics for BCSD. Ex2vec employs a novel pre-training strategy that enables the model to learn the impact of assembly instructions on register states, thus mitigating the reliance on learning the frequency and co-occurrence of the instructions in the assembly context. By simulating the execution of assembly instructions, Ex2Vec accurately captures the semantic features of assembly code, which is further demonstrated by Principal Component Analysis (PCA) that functionally similar instructions cluster closely in the embedding space. Extensive experiments on large datasets validate that Ex2vec performs exceptionally well in binary code similarity detection, surpassing all existing state-of-the-art methods. In real-world vulnerability detection experiments, Ex2Vec exhibits the highest accuracy.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"189 ","pages":"Article 107506"},"PeriodicalIF":6.0,"publicationDate":"2025-05-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143912253","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A spectral filtering approach to represent exemplars for visual few-shot classification","authors":"Tao Zhang ,&nbsp;Wu Huang","doi":"10.1016/j.neunet.2025.107498","DOIUrl":"10.1016/j.neunet.2025.107498","url":null,"abstract":"<div><div>Prototype is widely used to represent internal structure of category for few-shot learning, which was proposed as a simple inductive bias to address the issue of overfitting. However, for categories where prototypes do not exist or are difficult to represent, prototype representation may lead to underfitting, and these categories are better represented by exemplars. In this paper, we propose <span><math><mi>S</mi></math></span>hrinkage <span><math><mi>E</mi></math></span>xemplar <span><math><mi>Net</mi></math></span>works (SENet) for few-shot classification. In SENet, categories are represented by samples that shrink towards prototypes, appropriately describing both the presence and absence of prototypes. The shrinkage of samples is achieved through appropriate spectral filtering. Furthermore, a shrinkage exemplar loss is proposed to replace the widely used cross entropy loss for capturing the information of individual shrinkage samples. Several experiments were conducted on <em>mini</em>ImageNet, <em>tiered</em>-ImageNet and CIFAR-FS datasets. The experimental results demonstrate the effectiveness of our proposed method. The source code is publicly available at: <span><span>https://github.com/zhangtao2022/SENet</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"188 ","pages":"Article 107498"},"PeriodicalIF":6.0,"publicationDate":"2025-04-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143895236","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"On neural architecture search and hyperparameter optimization: A max-flow based approach","authors":"Chao Xue ,&nbsp;Jiaxing Li ,&nbsp;Xiaoxing Wang ,&nbsp;Yibing Zhan ,&nbsp;Junchi Yan ,&nbsp;Chun-Guang Li","doi":"10.1016/j.neunet.2025.107507","DOIUrl":"10.1016/j.neunet.2025.107507","url":null,"abstract":"<div><div>Automated Machine Learning (AutoML) involves the automatic production of models for specific tasks on given datasets, which can be divided into two aspects: Neural Architecture Search (NAS) for model construction and Hyperparameter Optimization (HPO) for model training. One of the most important components in an AutoML strategy is the search algorithm, which aims to recommend effective configurations according to historical observations. In this work, we propose a novel max-flow based search algorithm for AutoML by representing NAS and HPO as a Max-Flow problem on a graph and thus derive a couple of novel AutoML strategies, dubbed MF-NAS and MF-HPO, which handle the search space and the search strategy graphically. To be specific, MF-NAS induces parallel edges with capacities by combining different operations such as skip connections, convolutions, and pooling, whereas MF-HPO allows parallel edges to be regarded as intervals within the combined search spaces. The learned weights and capacities of the parallel edges are alternately updated during the search process. To make MF-NAS and MF-HPO more efficient, we implement a semi-synchronous search mode for NAS and a warmup scheme for HPO, respectively. We conduct extensive experiments to evaluate the competitive efficacy and efficiency of our proposed MF-NAS and MF-HPO across different datasets and search spaces.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"188 ","pages":"Article 107507"},"PeriodicalIF":6.0,"publicationDate":"2025-04-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143904237","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}