IF 6 1区计算机科学

Neural Networks Pub Date : 2025-03-01 Epub Date: 2024-12-01 DOI: 10.1016/j.neunet.2024.106970

Xiaojun Zhou, Chunna Zhao, Yaqun Huang, Chengli Zhou, Junjie Ye

{"title":"Improved fractional-order gradient descent method based on multilayer perceptron.","authors":"Xiaojun Zhou, Chunna Zhao, Yaqun Huang, Chengli Zhou, Junjie Ye","doi":"10.1016/j.neunet.2024.106970","DOIUrl":"10.1016/j.neunet.2024.106970","url":null,"abstract":"The fractional-order gradient descent (FOGD) method has been employed by numerous scholars in Artificial Neural Networks (ANN), with its superior performance validated both theoretically and experimentally. However, current FOGD methods only apply fractional-order differentiation to the loss function. The application of FOGD based on Autograd to hidden layers leverages the characteristics of fractional-order differentiation, significantly enhancing its flexibility. Moreover, the implementation of FOGD in the hidden layers serves as a necessary foundation for establishing a family of fractional-order deep learning optimizers, facilitating the widespread application of FOGD in deep learning. This paper proposes an improved fractional-order gradient descent (IFOGD) method based on Multilayer Perceptron (MLP). Firstly, a fractional matrix differentiation algorithm and its fractional matrix differentiation solver is proposed based on MLP, ensuring that IFOGD can be applied within the hidden layers. Subsequently, we overcome the issue of incorrect backpropagation direction caused by the absolute value symbol, ensuring that the IFOGD method does not cause divergence in the value of the loss function. Thirdly, fractional-order Autograd (FOAutograd) is proposed based on PyTorch by reconstructing Linear layer and Mean Squared Error Loss module. By combining FOAutograd with first-order adaptive deep learning optimizers, parameter matrices in each layer of ANN can be updated using fractional-order gradients. Finally, we compare and analyze the performance of IFOGD with other methods in simulation experiments and time series prediction tasks. The experimental results demonstrate that the IFOGD method exhibits performances.","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"183 ","pages":"106970"},"PeriodicalIF":6.0,"publicationDate":"2025-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142792265","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Advancements in exponential synchronization and encryption techniques: Quaternion-Valued Artificial Neural Networks with two-sided coefficients. 指数同步和加密技术的进展：具有双面系数的四元数值人工神经网络

IF 6 1区计算机科学

Neural Networks Pub Date : 2025-03-01 Epub Date: 2024-12-06 DOI: 10.1016/j.neunet.2024.106982

Chenyang Li, Kit Ian Kou, Yanlin Zhang, Yang Liu

{"title":"Advancements in exponential synchronization and encryption techniques: Quaternion-Valued Artificial Neural Networks with two-sided coefficients.","authors":"Chenyang Li, Kit Ian Kou, Yanlin Zhang, Yang Liu","doi":"10.1016/j.neunet.2024.106982","DOIUrl":"10.1016/j.neunet.2024.106982","url":null,"abstract":"This paper presents cutting-edge advancements in exponential synchronization and encryption techniques, focusing on Quaternion-Valued Artificial Neural Networks (QVANNs) that incorporate two-sided coefficients. The study introduces a novel approach that harnesses the Cayley-Dickson representation method to simplify the complex equations inherent in QVANNs, thereby enhancing computational efficiency by exploiting complex number properties. The study employs the Lyapunov theorem to craft a resilient control system, showcasing its exponential synchronization by skillfully regulating the Lyapunov function and its derivatives. This management ensures the stability and synchronization of the network, which is crucial for reliable performance in various applications. Extensive numerical simulations are conducted to substantiate the theoretical framework, providing empirical evidence supporting the presented design and proofs. Furthermore, the paper explores the practical application of QVANNs in the encryption and decryption of color images, showcasing the network's capability to handle complex data processing tasks efficiently. The findings of this research not only contribute significantly to the development of complex artificial neural networks but pave the way for further exploration into systems with diverse delay types.","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"183 ","pages":"106982"},"PeriodicalIF":6.0,"publicationDate":"2025-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142819819","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Event-based adaptive fixed-time optimal control for saturated fault-tolerant nonlinear multiagent systems via reinforcement learning algorithm. 基于强化学习算法的饱和容错非线性多智能体系统自适应定时最优控制。

IF 6 1区计算机科学

Neural Networks Pub Date : 2025-03-01 Epub Date: 2024-11-28 DOI: 10.1016/j.neunet.2024.106952

Huarong Yue, Jianwei Xia, Jing Zhang, Ju H Park, Xiangpeng Xie

{"title":"Event-based adaptive fixed-time optimal control for saturated fault-tolerant nonlinear multiagent systems via reinforcement learning algorithm.","authors":"Huarong Yue, Jianwei Xia, Jing Zhang, Ju H Park, Xiangpeng Xie","doi":"10.1016/j.neunet.2024.106952","DOIUrl":"10.1016/j.neunet.2024.106952","url":null,"abstract":"This article investigates the problem of adaptive fixed-time optimal consensus tracking control for nonlinear multiagent systems (MASs) affected by actuator faults and input saturation. To achieve optimal control, reinforcement learning (RL) algorithm which is implemented based on neural network (NN) is employed. Under the actor-critic structure, an innovative simple positive definite function is constructed to obtain the upper bound of the estimation error of the actor-critic NN updating law, which is crucial for analyzing fixed-time stabilization. Furthermore, auxiliary functions and estimation laws are designed to eliminate the coupling effects resulting from actuator faults and input saturation. Meanwhile, a novel event-triggered mechanism (ETM) that incorporates the consensus tracking errors into the threshold is proposed, thereby effectively conserving communication resources. Based on this, a fixed-time event-triggered control scheme grounded in RL is proposed through the integration of the backstepping technique and fixed-time theory. It is demonstrated that the consensus tracking errors converge to a specified range in a fixed time and all signals within the closed-loop systems are bounded. Finally, simulation results are provided to verify the effectiveness of the proposed control strategy.","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"183 ","pages":"106952"},"PeriodicalIF":6.0,"publicationDate":"2025-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142773950","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Delayed-feedback oscillators replicate the dynamics of multiplex networks: Wavefront propagation and stochastic resonance. 延迟反馈振荡器复制多路网络的动态：波前传播和随机共振。

IF 6 1区计算机科学

Neural Networks Pub Date : 2025-03-01 Epub Date: 2024-11-30 DOI: 10.1016/j.neunet.2024.106939

Anna Zakharova, Vladimir V Semenov

{"title":"Delayed-feedback oscillators replicate the dynamics of multiplex networks: Wavefront propagation and stochastic resonance.","authors":"Anna Zakharova, Vladimir V Semenov","doi":"10.1016/j.neunet.2024.106939","DOIUrl":"10.1016/j.neunet.2024.106939","url":null,"abstract":"The widespread development and use of neural networks have significantly enriched a wide range of computer algorithms and promise higher speed at lower cost. However, the imitation of neural networks by means of modern computing substrates is highly inefficient, whereas physical realization of large scale networks remains challenging. Fortunately, delayed-feedback oscillators, being much easier to realize experimentally, represent promising candidates for the empirical implementation of neural networks and next generation computing architectures. In the current research, we demonstrate that coupled bistable delayed-feedback oscillators emulate a multilayer network, where one single-layer network is connected to another single-layer network through coupling between replica nodes, i.e. the multiplex network. We show that all the aspects of the multiplexing impact on wavefront propagation and stochastic resonance identified in multilayer networks of bistable oscillators are entirely reproduced in the dynamics of time-delay oscillators. In particular, varying the coupling strength allows suppressing and enhancing the effect of stochastic resonance, as well as controlling the speed and direction of both deterministic and stochastic wavefront propagation. All the considered effects are studied in numerical simulations and confirmed in physical experiments, showing an excellent correspondence and disclosing thereby the robustness of the observed phenomena.","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"183 ","pages":"106939"},"PeriodicalIF":6.0,"publicationDate":"2025-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142787484","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Cognitive process and information processing model based on deep learning algorithms. 基于深度学习算法的认知过程和信息处理模型。

IF 6 1区计算机科学

Neural Networks Pub Date : 2025-03-01 Epub Date: 2024-12-02 DOI: 10.1016/j.neunet.2024.106999

DongCai Zhao

引用次数: 0

PDG2Seq: Periodic Dynamic Graph to Sequence Model for Traffic Flow Prediction. PDG2Seq：交通流预测的周期动态图到序列模型。

IF 6 1区计算机科学

Neural Networks Pub Date : 2025-03-01 Epub Date: 2024-12-02 DOI: 10.1016/j.neunet.2024.106941

Jin Fan, Wenchao Weng, Qikai Chen, Huifeng Wu, Jia Wu

{"title":"PDG2Seq: Periodic Dynamic Graph to Sequence Model for Traffic Flow Prediction.","authors":"Jin Fan, Wenchao Weng, Qikai Chen, Huifeng Wu, Jia Wu","doi":"10.1016/j.neunet.2024.106941","DOIUrl":"10.1016/j.neunet.2024.106941","url":null,"abstract":"Traffic flow prediction is the foundation of intelligent traffic management systems. Current methods prioritize the development of intricate models to capture spatio-temporal correlations, yet they often neglect the exploitation of latent features within traffic flow. Firstly, the correlation among different road nodes exhibits dynamism rather than remaining static. Secondly, traffic data exhibits evident periodicity, yet current research lacks the exploration and utilization of periodic features. Lastly, current models typically rely solely on historical data for modeling, resulting in the limitation of accurately capturing future trend changes in traffic flow. To address these findings, this paper proposes a Periodic Dynamic Graph to Sequence Model (PDG2Seq) for traffic flow prediction. PDG2Seq consists of the Periodic Feature Selection Module (PFSM) and the Periodic Dynamic Graph Convolutional Gated Recurrent Unit (PDCGRU) to further extract the spatio-temporal features of the dynamic real-time traffic. The PFSM extracts learned periodic features using time points as indices, while the PDCGRU leverages the extracted periodic features from the PFSM and dynamic features from traffic flow to generate a Periodic Dynamic Graph for extracting spatio-temporal features. In the decoding phase, PDG2Seq utilizes periodic features corresponding to the prediction target to capture future trend changes, leading to more accurate predictions. Comprehensive experiments conducted on four large-scale datasets substantiate the superiority of PDG2Seq over existing state-of-the-art baselines. Related codes are available at https://github.com/wengwenchao123/PDG2Seq.","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"183 ","pages":"106941"},"PeriodicalIF":6.0,"publicationDate":"2025-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142792291","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Heterogeneous Graph Embedding with Dual Edge Differentiation. 基于双边缘微分的异构图嵌入。

IF 6 1区计算机科学

Neural Networks Pub Date : 2025-03-01 Epub Date: 2024-12-03 DOI: 10.1016/j.neunet.2024.106965

Yuhong Chen, Fuhai Chen, Zhihao Wu, Zhaoliang Chen, Zhiling Cai, Yanchao Tan, Shiping Wang

{"title":"Heterogeneous Graph Embedding with Dual Edge Differentiation.","authors":"Yuhong Chen, Fuhai Chen, Zhihao Wu, Zhaoliang Chen, Zhiling Cai, Yanchao Tan, Shiping Wang","doi":"10.1016/j.neunet.2024.106965","DOIUrl":"10.1016/j.neunet.2024.106965","url":null,"abstract":"Recently, heterogeneous graphs have attracted widespread attention as a powerful and practical superclass of traditional homogeneous graphs, which reflect the multi-type node entities and edge relations in the real world. Most existing methods adopt meta-path construction as the mainstream to learn long-range heterogeneous semantic messages between nodes. However, such schema constructs the node-wise correlation by connecting nodes via pre-computed fixed paths, which neglects the diversities of meta-paths on the path type and path range. In this paper, we propose a meta-path-based semantic embedding schema, which is called Heterogeneous Graph Embedding with Dual Edge Differentiation (HGE-DED) to adequately construct flexible meta-path combinations thus learning the rich and discriminative semantic of target nodes. Concretely, HGE-DED devises a Multi-Type and multi-Range Meta-Path Construction (MTR-MP Construction), which covers the comprehensive exploration of meta-path combinations from path type and path range, expressing the diversity of edges at more fine-grained scales. Moreover, HGE-DED designs the semantics and meta-path joint guidance, constructing a hierarchical short- and long-range relation adjustment, which constrains the path learning as well as minimizes the impact of edge heterophily on heterogeneous graphs. Experimental results on four benchmark datasets demonstrate the effectiveness of HGE-DED compared with state-of-the-art methods.","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"183 ","pages":"106965"},"PeriodicalIF":6.0,"publicationDate":"2025-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142814706","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

M4Net: Multi-level multi-patch multi-receptive multi-dimensional attention network for infrared small target detection. M4Net：用于红外小目标探测的多层次多补丁多接受多维关注网络。

IF 6 1区计算机科学

Neural Networks Pub Date : 2025-03-01 Epub Date: 2024-12-05 DOI: 10.1016/j.neunet.2024.107026

Fan Zhang, Huilin Hu, Biyu Zou, Meizu Luo

{"title":"M4Net: Multi-level multi-patch multi-receptive multi-dimensional attention network for infrared small target detection.","authors":"Fan Zhang, Huilin Hu, Biyu Zou, Meizu Luo","doi":"10.1016/j.neunet.2024.107026","DOIUrl":"10.1016/j.neunet.2024.107026","url":null,"abstract":"The detection of infrared small targets is getting more and more attention, and has a wider application in both military and civilian fields. The traditional infrared small target detection methods heavily rely on the setting of manual features, and the deep learning-based method easily lose the targets in deep layers due to several downsampling operations. To handle this problem, we design multi-level multi-patch multi-receptive multi-dimensional attention network (M4Net) to achieve information interaction among high-level and low-level features for maintaining target contour and location detail. Multi-level feature extraction module (MFEM) with multilayer vision transformer (ViT) is introduced under the encoder-decoder framework to fuse multi-scale features. Multi-patch attention module (MPAM) and multi-receptive field module (MRFM) are proposed to capture and enhance the feature information. Multi-dimension interactive module (MDIM) is designed to connect the attention mechanism on multiscale features to enhance the network's leaning ability. Finally, the extensive experiments carried out on infrared small target detection dataset demonstrate that our method achieves better performance compared to other methods.","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"183 ","pages":"107026"},"PeriodicalIF":6.0,"publicationDate":"2025-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142808458","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

ST-Tree with interpretability for multivariate time series classification. st树与解释性多变量时间序列分类。

IF 6 1区计算机科学

Neural Networks Pub Date : 2025-03-01 Epub Date: 2024-12-03 DOI: 10.1016/j.neunet.2024.106951

Mingsen Du, Yanxuan Wei, Yingxia Tang, Xiangwei Zheng, Shoushui Wei, Cun Ji

{"title":"ST-Tree with interpretability for multivariate time series classification.","authors":"Mingsen Du, Yanxuan Wei, Yingxia Tang, Xiangwei Zheng, Shoushui Wei, Cun Ji","doi":"10.1016/j.neunet.2024.106951","DOIUrl":"10.1016/j.neunet.2024.106951","url":null,"abstract":"Multivariate time series classification is of great importance in practical applications and is a challenging task. However, deep neural network models such as Transformers exhibit high accuracy in multivariate time series classification but lack interpretability and fail to provide insights into the decision-making process. On the other hand, traditional approaches based on decision tree classifiers offer clear decision processes but relatively lower accuracy. Swin Transformer (ST) addresses these issues by leveraging self-attention mechanisms to capture both fine-grained local patterns and global patterns. It can also model multi-scale feature representation learning, thereby providing a more comprehensive representation of time series features. To tackle the aforementioned challenges, we propose ST-Tree with interpretability for multivariate time series classification. Specifically, the ST-Tree model combines ST as the backbone network with an additional neural tree model. This integration allows us to fully leverage the advantages of ST in learning time series context while providing interpretable decision processes through the neural tree. This enables researchers to gain clear insights into the model's decision-making process and extract meaningful interpretations. Through experimental evaluations on 10 UEA datasets, we demonstrate that the ST-Tree model improves accuracy in multivariate time series classification tasks and provides interpretability through visualizing the decision-making process across different datasets.","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"183 ","pages":"106951"},"PeriodicalIF":6.0,"publicationDate":"2025-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142796537","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Semantic Mask Reconstruction and Category Semantic Learning for few-shot image generation. 基于语义掩码重构和类别语义学习的少镜头图像生成。

IF 6 1区计算机科学

Neural Networks Pub Date : 2025-03-01 Epub Date: 2024-12-03 DOI: 10.1016/j.neunet.2024.106946

Ting Xiao, Yunjie Cai, Jiaoyan Guan, Zhe Wang

{"title":"Semantic Mask Reconstruction and Category Semantic Learning for few-shot image generation.","authors":"Ting Xiao, Yunjie Cai, Jiaoyan Guan, Zhe Wang","doi":"10.1016/j.neunet.2024.106946","DOIUrl":"10.1016/j.neunet.2024.106946","url":null,"abstract":"Few-shot image generation aims at generating novel images for the unseen category when given K images from the same category. Despite significant advancements in existing few-shot image generation methods, great challenges remain regarding the quality and diversity of the generated images. This issue stems from the model's struggle to fully comprehend the semantic content of images and extract sufficiently semantic representations. To address these issues, we propose a semantic mask reconstruction (SMR) and category semantic learning (CSL) method for few-shot image generation. Specifically, SMR performs mask reconstruction in a high-level semantic space and designs a strategy for dynamically adjusting the mask ratio, which increases the difficulty of the generation tasks by gradually increasing the mask ratio to enhance the learning ability of the discriminator, thereby prompting the generator to learn more critical features relevant to the generation task. In addition, CSL introduces a triplet loss to optimize the distance between the generated image, its corresponding input image, and input images of other categories. This encourages the generative model to discern subtle differences between categories, thereby achieving more fine-grained generation and improving the fidelity of generated images. Both SMR and CSL can function as plug-and-play modules. Extensive experimental results across three standard datasets demonstrate that the SMR-CSL outperforms other methods in terms of the quality and diversity of the generated images. Furthermore, the results of downstream classification experiments verify that the images generated by the proposed method can effectively assist downstream classification tasks.","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"183 ","pages":"106946"},"PeriodicalIF":6.0,"publicationDate":"2025-03-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142792406","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

计算机科学最新文献