Neural Networks最新文献

{"title":"PrediRep: Modeling hierarchical predictive coding with an unsupervised deep learning network","authors":"Ibrahim C. Hashim,&nbsp;Mario Senden,&nbsp;Rainer Goebel","doi":"10.1016/j.neunet.2025.107246","DOIUrl":"10.1016/j.neunet.2025.107246","url":null,"abstract":"<div><div>Hierarchical predictive coding (hPC) provides a compelling framework for understanding how the cortex predicts future sensory inputs by minimizing prediction errors through an internal generative model of the external world. Existing deep learning models inspired by hPC incorporate architectural choices that deviate from core hPC principles, potentially limiting their utility for neuroscientific investigations. We introduce PrediRep (Predicting Representations), a novel deep learning network that adheres more closely to architectural principles of hPC. We validate PrediRep by comparing its functional alignment with hPC to that of existing models after being trained on a next-frame prediction task. Our findings demonstrate that PrediRep, particularly when trained with an all-level loss function (PrediRepAll), exhibits high functional alignment with hPC. In contrast to other contemporary deep learning networks inspired by hPC, it consistently processes input-relevant information at higher hierarchical levels and maintains active representations and accurate predictions across all hierarchical levels. Although PrediRep was designed primarily to serve as a model suitable for neuroscientific research rather than to optimize performance, it nevertheless achieves competitive performance in next-frame prediction while utilizing significantly fewer trainable parameters than alternative models. Our results underscore that even minor architectural deviations from neuroscientific theories like hPC can lead to significant functional discrepancies. By faithfully adhering to hPC principles, PrediRep provides a more accurate tool for in silico exploration of cortical phenomena. PrediRep’s lightweight and biologically plausible design makes it well-suited for future studies aiming to investigate the neural underpinnings of predictive coding and to derive empirically testable predictions.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"185 ","pages":"Article 107246"},"PeriodicalIF":6.0,"publicationDate":"2025-02-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143388113","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Bio-inspired two-stage network for efficient RGB-D salient object detection","authors":"Peng Ren ,&nbsp;Tian Bai ,&nbsp;Fuming Sun","doi":"10.1016/j.neunet.2025.107244","DOIUrl":"10.1016/j.neunet.2025.107244","url":null,"abstract":"<div><div>Recently, with the development of the Convolutional Neural Network and Vision Transformer, the detection accuracy of the RGB-D salient object detection (SOD) model has been greatly improved. However, most of the existing methods cannot balance computational efficiency and performance well. In this paper, inspired by the P visual pathway and the M visual pathway in the primate biological visual system, we propose a Bio-inspired Two-stage Network for Efficient RGB-D SOD, named BTNet. It simulates the visual information processing of the P visual pathway and the M visual pathway. Specifically, BTNet contains two stages: region locking and object refinement. Among them, the region locking stage simulates the visual information processing process of the M visual pathway to obtain coarse-grained visual representation. The object refinement stage simulates the visual information processing process of the P visual pathway to obtain fine-grained visual representation. Experimental results show that BTNet outperforms other state-of-the-art methods on six mainstream benchmark datasets, achieving significant parameter reduction and processing 384 × 384 resolution images at a speed of 175.4 Frames Per Second (FPS). Compared with the cutting-edge method CPNet, BTNet reduces parameters by 93.6% and is nearly 7.2 times faster. The source codes are available at <span><span>https://github.com/ROC-Star/BTNet</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"185 ","pages":"Article 107244"},"PeriodicalIF":6.0,"publicationDate":"2025-02-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143377578","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Robust deep learning from weakly dependent data","authors":"William Kengne ,&nbsp;Modou Wade","doi":"10.1016/j.neunet.2025.107227","DOIUrl":"10.1016/j.neunet.2025.107227","url":null,"abstract":"<div><div>Recent developments on deep learning established some theoretical properties of deep neural networks estimators. However, most of the existing works on this topic are restricted to bounded loss functions or (sub)-Gaussian or bounded variables. This paper considers robust deep learning from weakly dependent observations, with unbounded loss function and unbounded output. It is only assumed that the output variable has a finite <span><math><mi>r</mi></math></span> order moment, with <span><math><mrow><mi>r</mi><mo>&gt;</mo><mn>1</mn></mrow></math></span>. Non asymptotic bounds for the expected excess risk of the deep neural network estimator are established under strong mixing, and <span><math><mi>ψ</mi></math></span>-weak dependence assumptions on the observations. We derive a relationship between these bounds and <span><math><mi>r</mi></math></span>, and when the data have moments of any order, the convergence rate is close to some well-known results. When the target predictor belongs to the class of Hölder smooth functions with sufficiently large smoothness index, the rate of the expected excess risk for exponentially strongly mixing data is close to that obtained with i.i.d. samples. Application to robust nonparametric regression and robust nonparametric autoregression are considered. The simulation study for models with heavy-tailed errors shows that, robust estimators with absolute loss and Huber loss function outperform the least squares method.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"185 ","pages":"Article 107227"},"PeriodicalIF":6.0,"publicationDate":"2025-02-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143377653","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Generative and contrastive graph representation learning with message passing","authors":"Ying Tang,&nbsp;Yining Yang,&nbsp;Guodao Sun","doi":"10.1016/j.neunet.2025.107224","DOIUrl":"10.1016/j.neunet.2025.107224","url":null,"abstract":"<div><div>Self-supervised graph representation learning (SSGRL) has emerged as a promising approach for graph embeddings because it does not rely on manual labels. SSGRL methods are generally divided into generative and contrastive approaches. Generative methods often suffer from poor graph quality, while contrastive methods, which compare augmented views, are more resistant to noise. However, the performance of contrastive methods depends heavily on well-designed data augmentation and high-quality negative samples. Pure generative or contrastive methods alone cannot balance both robustness and performance. To address these issues, we propose a self-supervised graph representation learning method that integrates generative and contrastive ideas, namely Contrastive Generative Message Passing Graph Learning (CGMP-GL). CGMP-GL incorporates the concept of contrast into the generative model and message aggregation module, enhancing the discriminability of node representations by aligning positive samples and separating negative samples. On one hand, CGMP-GL integrates multi-granularity topology and feature information through cross-view multi-level contrast while reconstructing masked node features. On the other hand, CGMP-GL optimizes node representations through self-supervised contrastive message passing, thereby enhancing model performance in various downstream tasks. Extensive experiments over multiple datasets and downstream tasks demonstrate the effectiveness and robustness of CGMP-GL.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"185 ","pages":"Article 107224"},"PeriodicalIF":6.0,"publicationDate":"2025-02-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143349462","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"GARNN: An interpretable graph attentive recurrent neural network for predicting blood glucose levels via multivariate time series","authors":"Chengzhe Piao ,&nbsp;Taiyu Zhu ,&nbsp;Stephanie E. Baldeweg ,&nbsp;Paul Taylor ,&nbsp;Pantelis Georgiou ,&nbsp;Jiahao Sun ,&nbsp;Jun Wang ,&nbsp;Kezhi Li","doi":"10.1016/j.neunet.2025.107229","DOIUrl":"10.1016/j.neunet.2025.107229","url":null,"abstract":"<div><div>Accurate prediction of future blood glucose (BG) levels can effectively improve BG management for people living with type 1 or 2 diabetes, thereby reducing complications and improving quality of life. The state of the art of BG prediction has been achieved by leveraging advanced deep learning methods to model multimodal data, i.e., sensor data and self-reported event data, organized as multi-variate time series (MTS). However, these methods are mostly regarded as “black boxes” and not entirely trusted by clinicians and patients. In this paper, we propose interpretable graph attentive recurrent neural networks (GARNNs) to model MTS, explaining variable contributions via summarizing variable importance and generating feature maps by graph attention mechanisms instead of post-hoc analysis. We evaluate GARNNs on four datasets, representing diverse clinical scenarios. Upon comparison with fifteen well-established baseline methods, GARNNs not only achieve the best prediction accuracy but also provide high-quality temporal interpretability, in particular for postprandial glucose levels as a result of corresponding meal intake and insulin injection. These findings underline the potential of GARNN as a robust tool for improving diabetes care, bridging the gap between deep learning technology and real-world healthcare solutions.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"185 ","pages":"Article 107229"},"PeriodicalIF":6.0,"publicationDate":"2025-02-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143372942","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Multi-level social network alignment via adversarial learning and graphlet modeling","authors":"Jingyuan Duan ,&nbsp;Zhao Kang ,&nbsp;Ling Tian ,&nbsp;Yichen Xin","doi":"10.1016/j.neunet.2025.107230","DOIUrl":"10.1016/j.neunet.2025.107230","url":null,"abstract":"<div><div>Aiming to identify corresponding users in different networks, social network alignment is significant for numerous subsequent applications. Most existing models apply consistency assumptions on undirected networks, ignoring platform disparity caused by diverse functionalities and universal directed relations like follower–followee. Due to indistinguishable nodes and relations, subgraph isomorphism is also unavoidable in neighborhoods. In order to precisely align directed and attributed social networks, we propose the Multi-level Adversarial and Graphlet-based Social Network Alignment (MAGSNA), which unifies networks as a whole at individual-level and learns discriminative graphlet-based features at partition-level simultaneously, thereby alleviating both platform disparity and subgraph isomorphism. Specifically, at individual-level, we relieve topology disparity by the random walk with restart, while developing directed weight-sharing network embeddings and a bidirectional optimizer on Wasserstein graph adversarial networks for attribute disparity. At partition-level, we extract overlapped partitions from graphlet orbits, then design weight-sharing partition embeddings and a hubness-aware refinement to derive discriminative features. By fusing the similarities of these two levels, we obtain a precise and thorough alignment. Experiments on real-world and synthetic datasets demonstrate that MAGSNA outperforms state-of-the-art methods, exhibiting competitive efficiency and superior robustness.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"185 ","pages":"Article 107230"},"PeriodicalIF":6.0,"publicationDate":"2025-02-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143349988","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Fuzzy spatiotemporal event-triggered control for the synchronization of IT2 T–S fuzzy CVRDNNs with mini-batch machine learning supervision","authors":"Shuoting Wang ,&nbsp;Kaibo Shi ,&nbsp;Jinde Cao ,&nbsp;Shiping Wen","doi":"10.1016/j.neunet.2025.107220","DOIUrl":"10.1016/j.neunet.2025.107220","url":null,"abstract":"<div><div>This paper is centered on the development of a fuzzy memory-based spatiotemporal event-triggered mechanism (FMSETM) for the synchronization of the drive-response interval type-2 (IT2) Takagi–Sugeno (T–S) fuzzy complex-valued reaction–diffusion neural networks (CVRDNNs). CVRDNNs have a higher processing capability and can perform better than multilayered real-valued RDNNs. Firstly, a general IT2 T–S fuzzy neural network model is constructed by considering complex-valued parameters and the reaction–diffusion terms. Secondly, a mini-batch semi-stochastic machine learning technique is proposed to optimize the maximum sampling period in an FMSETM. Furthermore, by constructing an asymmetric Lyapunov functional (LF) dependent on the membership function (MF), certain symmetric and positive-definite constraints of matrices are removed. The synchronization criteria are derived via linear matrix inequalities (LMIs) for the IT2 T–S fuzzy CVRDNNs. Finally, two numerical examples are utilized to corroborate the feasibility of the developed approach. From the simulation results, it can be seen that introducing machine learning techniques into the synchronization problem of CVRDNNs can improve the efficiency of convergence.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"185 ","pages":"Article 107220"},"PeriodicalIF":6.0,"publicationDate":"2025-02-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143377654","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"RotInv-PCT: Rotation-Invariant Point Cloud Transformer via feature separation and aggregation","authors":"Cheng He,&nbsp;Zhenjie Zhao,&nbsp;Xuebo Zhang,&nbsp;Hang Yu,&nbsp;Runhua Wang","doi":"10.1016/j.neunet.2025.107223","DOIUrl":"10.1016/j.neunet.2025.107223","url":null,"abstract":"<div><div>The widespread use of point clouds has spurred the rapid development of neural networks for point cloud processing. A crucial property of these networks is maintaining consistent output results under random rotations of the input point cloud, namely, rotation invariance. The dominant approach achieves rotation invariance is to construct local coordinate systems for computing invariant local point cloud coordinates. However, this method neglects the relative pose relationships between local point cloud structures, leading to a decline in network performance. To address this limitation, we propose a novel Rotation-Invariant Point Cloud Transformer (RotInv-PCT). This method extracts the local abstract shape features of the point cloud using Local Reference Frames (LRFs) and explicitly computes the spatial relative pose features between local point clouds, both of which are proven to be rotation-invariant. Furthermore, to capture the long-range pose dependencies between points, we introduce an innovative Feature Aggregation Transformer (FAT) model, which seamlessly fuses the pose features with the shape features to obtain a globally rotation-invariant representation. Moreover, to manage large-scale point clouds, we utilize hierarchical random downsampling to gradually decrease the scale of point clouds, followed by feature aggregation through FAT. To demonstrate the effectiveness of RotInv-PCT, we conducted comparative experiments across various tasks and datasets, including point cloud classification on ScanObjectNN and ModelNet40, part segmentation on ShapeNet, and semantic segmentation on S3DIS and KITTI. Thanks to our provable rotation-invariant features and FAT, our method generally outperforms state-of-the-art networks. In particular, we highlight that RotInv-PCT achieved a 2% improvement in real-world point cloud classification tasks compared to the strongest baseline. Furthermore, in the semantic segmentation task, we improved the performance on the S3DIS dataset by 10% and, for the first time, realized rotation-invariant point cloud semantic segmentation on the KITTI dataset.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"185 ","pages":"Article 107223"},"PeriodicalIF":6.0,"publicationDate":"2025-02-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143349464","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Fast finite-time quantized control of multi-layer networks and its applications in secure communication","authors":"Qian Tang ,&nbsp;Shaocheng Qu ,&nbsp;Wei Zheng ,&nbsp;Zhengwen Tu","doi":"10.1016/j.neunet.2025.107225","DOIUrl":"10.1016/j.neunet.2025.107225","url":null,"abstract":"<div><div>This paper introduces a quantized controller to address the challenge of fast finite-time synchronization of multi-layer networks, where each layer represents a distinct type of interaction within complex systems. Firstly, based on the stability theory, a novel fast finite-time stability criterion is derived, which can set a smaller upper limit of the settling time by comparing it with the general finite-time stability. Secondly, by converting continuous error signals into piecewise continuous forms, a quantized control scheme is employed to realize fast finite-time synchronization in multi-layer networks, which can save control resources and alleviate communication congestion. Finally, the feasibility of the quantized control algorithm in multi-layer network synchronization and its applications in secure communication are verified through numerical simulation.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"185 ","pages":"Article 107225"},"PeriodicalIF":6.0,"publicationDate":"2025-02-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143349990","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"TF-BERT: Tensor-based fusion BERT for multimodal sentiment analysis","authors":"Jingming Hou ,&nbsp;Nazlia Omar ,&nbsp;Sabrina Tiun ,&nbsp;Saidah Saad ,&nbsp;Qian He","doi":"10.1016/j.neunet.2025.107222","DOIUrl":"10.1016/j.neunet.2025.107222","url":null,"abstract":"<div><div>Multimodal Sentiment Analysis (MSA) has gained significant attention due to the limitations of unimodal sentiment recognition in complex real-world applications. Traditional approaches typically focus on using the Transformer for fusion. However, these traditional approaches often fall short because the Transformer can only process two modalities simultaneously, leading to insufficient information exchange and potential loss of emotional data. To address the limitation of traditional Crossmodal Transformer models, which can only process two modalities at a time, we propose a novel Tensor-based Fusion BERT model (TF-BERT). The core of TF-BERT is the Tensor-based Crossmodal Fusion (TCF) module, which seamlessly integrates into the pre-trained BERT language model. By embedding the TCF module into multiple layers of BERT’s Transformer, we progressively achieve dynamic complementation between different modalities. Additionally, we designed the Tensor-based Crossmodal Transformer (TCT) module, which introduces a tensor-based Transformer mechanism capable of simultaneously processing three different modalities. This allows for comprehensive information exchange between the target modality and the other two source modalities, thus strengthening the representation of the target modality. The TCT overcomes the limitation of existing the Crossmodal Transformer structures, which can only handle relationships between two modalities. Furthermore, to validate the effectiveness of TF-BERT, we conducted extensive experiments on the CMU-MOSI and CMU-MOSEI datasets. TF-BERT not only achieved the best results across most metrics compared to recent state-of-the-art baselines, but also demonstrated the effectiveness of its two modules through ablation studies. The findings suggest that TF-BERT effectively addresses the limitations of previous models by progressively integrating and simultaneously capturing complex emotional interactions across all modalities.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"185 ","pages":"Article 107222"},"PeriodicalIF":6.0,"publicationDate":"2025-02-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143379414","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}