Neural Networks最新文献

{"title":"Contrastive fine-grained domain adaptation network for EEG-based vigilance estimation.","authors":"Kangning Wang, Wei Wei, Weibo Yi, Shuang Qiu, Huiguang He, Minpeng Xu, Dong Ming","doi":"10.1016/j.neunet.2024.106617","DOIUrl":"10.1016/j.neunet.2024.106617","url":null,"abstract":"<p><p>Vigilance state is crucial for the effective performance of users in brain-computer interface (BCI) systems. Most vigilance estimation methods rely on a large amount of labeled data to train a satisfactory model for the specific subject, which limits the practical application of the methods. This study aimed to build a reliable vigilance estimation method using a small amount of unlabeled calibration data. We conducted a vigilance experiment in the designed BCI-based cursor-control task. Electroencephalogram (EEG) signals of eighteen participants were recorded in two sessions on two different days. And, we proposed a contrastive fine-grained domain adaptation network (CFGDAN) for vigilance estimation. Here, an adaptive graph convolution network (GCN) was built to project the EEG data of different domains into a common space. The fine-grained feature alignment mechanism was designed to weight and align the feature distributions across domains at the EEG channel level, and the contrastive information preservation module was developed to preserve the useful target-specific information during the feature alignment. The experimental results show that the proposed CFGDAN outperforms the compared methods in our BCI vigilance dataset and SEED-VIG dataset. Moreover, the visualization results demonstrate the efficacy of the designed feature alignment mechanisms. These results indicate the effectiveness of our method for vigilance estimation. Our study is helpful for reducing calibration efforts and promoting the practical application potential of vigilance estimation methods.</p>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":null,"pages":null},"PeriodicalIF":6.0,"publicationDate":"2024-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142057086","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Joint computation offloading and resource allocation for end-edge collaboration in internet of vehicles via multi-agent reinforcement learning.","authors":"Cong Wang, Yaoming Wang, Ying Yuan, Sancheng Peng, Guorui Li, Pengfei Yin","doi":"10.1016/j.neunet.2024.106621","DOIUrl":"10.1016/j.neunet.2024.106621","url":null,"abstract":"<p><p>Vehicular edge computing (VEC), a promising paradigm for the development of emerging intelligent transportation systems, can provide lower service latency for vehicular applications. However, it is still a challenge to fulfill the requirements of such applications with stringent latency requirements in the VEC system with limited resources. In addition, existing methods focus on handling the offloading task in a certain time slot with statically allocated resources, but ignore the heterogeneous tasks' different resource requirements, resulting in resource wastage. To solve the real-time task offloading and heterogeneous resource allocation problem in VEC system, we propose a decentralized solution based on the attention mechanism and recurrent neural networks (RNN) with a multi-agent distributed deep deterministic policy gradient (AR-MAD4PG). First, to address the partial observability of agents, we construct a shared agent graph and propose a periodic communication mechanism that enables edge nodes to aggregate information from other edge nodes. Second, to help agents better understand the current system state, we design an RNN-based feature extraction network to capture the historical state and resource allocation information of the VEC system. Thirdly, to tackle the challenges of excessive joint observation-action space and ineffective information interference, we adopt the multi-head attention mechanism to compress the dimension of the observation-action space of agents. Finally, we build a simulation model based on the actual vehicle trajectories, and the experimental results show that our proposed method outperforms the existing approaches.</p>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":null,"pages":null},"PeriodicalIF":6.0,"publicationDate":"2024-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141996788","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"An information-theoretic perspective of physical adversarial patches.","authors":"Bilel Tarchoun, Anouar Ben Khalifa, Mohamed Ali Mahjoub, Nael Abu-Ghazaleh, Ihsen Alouani","doi":"10.1016/j.neunet.2024.106590","DOIUrl":"10.1016/j.neunet.2024.106590","url":null,"abstract":"<p><p>Real-world adversarial patches were shown to be successful in compromising state-of-the-art models in various computer vision applications. Most existing defenses rely on analyzing input or feature level gradients to detect the patch. However, these methods have been compromised by recent GAN-based attacks that generate naturalistic patches. In this paper, we propose a new perspective to defend against adversarial patches based on the entropy carried by the input, rather than on its saliency. We present Jedi, a new defense against adversarial patches that tackles the patch localization problem from an information theory perspective; leveraging the high entropy of adversarial patches to identify potential patch zones, and using an autoencoder to complete patch regions from high entropy kernels. Jedi achieves high-precision adversarial patch localization and removal, detecting on average 90% of adversarial patches across different benchmarks, and recovering up to 94% of successful patch attacks. Since Jedi relies on an input entropy analysis, it is model-agnostic, and can be applied to off-the-shelf models without changes to the training or inference of the models. Moreover, we propose a comprehensive qualitative analysis that investigates the cases where Jedi fails, comparatively with related methods. Interestingly, we find a significant core failure cases among the different defenses share one common property: high entropy. We think that this work offers a new perspective to understand the adversarial effect under physical-world settings. We also leverage these findings to enhance Jedi's handling of entropy outliers by introducing Adaptive Jedi, which boosts performance by up to 9% in challenging images.</p>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":null,"pages":null},"PeriodicalIF":6.0,"publicationDate":"2024-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142009846","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Multi-focus image fusion with parameter adaptive dual channel dynamic threshold neural P systems.","authors":"Bo Li, Lingling Zhang, Jun Liu, Hong Peng, Qianying Wang, Jiaqi Liu","doi":"10.1016/j.neunet.2024.106603","DOIUrl":"10.1016/j.neunet.2024.106603","url":null,"abstract":"<p><p>Multi-focus image fusion (MFIF) is an important technique that aims to combine the focused regions of multiple source images into a fully clear image. Decision-map methods are widely used in MFIF to maximize the preservation of information from the source images. While many decision-map methods have been proposed, they often struggle with difficulties in determining focus and non-focus boundaries, further affecting the quality of the fused images. Dynamic threshold neural P (DTNP) systems are computational models inspired by biological spiking neurons, featuring dynamic threshold and spiking mechanisms to better distinguish focused and unfocused regions for decision map generation. However, original DTNP systems require manual parameter configuration and have only one stimulus. Therefore, they are not suitable to be used directly for generating high-precision decision maps. To overcome these limitations, we propose a variant called parameter adaptive dual channel DTNP (PADCDTNP) systems. Inspired by the spiking mechanisms of PADCDTNP systems, we further develop a new MFIF method. As a new neural model, PADCDTNP systems adaptively estimate parameters according to multiple external inputs to produce decision maps with robust boundaries, resulting in high-quality fusion results. Comprehensive experiments on the Lytro/MFFW/MFI-WHU dataset show that our method achieves advanced performance and yields comparable results to the fourteen representative MFIF methods. In addition, compared to the standard DTNP systems, PADCDTNP systems improve the fusion performance and fusion efficiency on the three datasets by 5.69% and 86.03%, respectively. The codes for both the proposed method and the comparison methods are released at https://github.com/MorvanLi/MFIF-PADCDTNP.</p>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":null,"pages":null},"PeriodicalIF":6.0,"publicationDate":"2024-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141989374","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Coagulo-Net: Enhancing the mathematical modeling of blood coagulation using physics-informed neural networks","authors":"","doi":"10.1016/j.neunet.2024.106732","DOIUrl":"10.1016/j.neunet.2024.106732","url":null,"abstract":"<div><p>Blood coagulation, which involves a group of complex biochemical reactions, is a crucial step in hemostasis to stop bleeding at the injury site of a blood vessel. Coagulation abnormalities, such as hypercoagulation and hypocoagulation, could either cause thrombosis or hemorrhage, resulting in severe clinical consequences. Mathematical models of blood coagulation have been widely used to improve the understanding of the pathophysiology of coagulation disorders, guide the design and testing of new anticoagulants or other therapeutic agents, and promote precision medicine. However, estimating the parameters in these coagulation models has been challenging as not all reaction rate constants and new parameters derived from model assumptions are measurable. Although various conventional methods have been employed for parameter estimation for coagulation models, the existing approaches have several shortcomings. Inspired by the physics-informed neural networks, we propose Coagulo-Net, which synergizes the strengths of deep neural networks with the mechanistic understanding of the blood coagulation processes to enhance the mathematical models of the blood coagulation cascade. We assess the performance of the Coagulo-Net using two existing coagulation models with different extents of complexity. Our simulation results illustrate that Coagulo-Net can efficiently infer the unknown model parameters and dynamics of species based on sparse measurement data and data contaminated with noise. In addition, we show that Coagulo-Net can process a mixture of synthetic and experimental data and refine the predictions of existing mathematical models of coagulation. These results demonstrate the promise of Coagulo-Net in enhancing current coagulation models and aiding the creation of novel models for physiological and pathological research. These results showcase the potential of Coagulo-Net to advance computational modeling in the study of blood coagulation, improving both research methodologies and the development of new therapies for treating patients with coagulation disorders.</p></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":null,"pages":null},"PeriodicalIF":6.0,"publicationDate":"2024-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142271787","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Decoupling visual and identity features for adversarial palm-vein image attack","authors":"","doi":"10.1016/j.neunet.2024.106693","DOIUrl":"10.1016/j.neunet.2024.106693","url":null,"abstract":"<div><p>Palm-vein has been widely used for biometric recognition due to its resistance to theft and forgery. However, with the emergence of adversarial attacks, most existing palm-vein recognition methods are vulnerable to adversarial image attacks, and to the best of our knowledge, there is still no study specifically focusing on palm-vein image attacks. In this paper, we propose an adversarial palm-vein image attack network that generates highly similar adversarial palm-vein images to the original samples, but with altered palm-identities. Unlike most existing generator-oriented methods that directly learn image features via concatenated convolutional layers, our proposed network first maps palm-vein images into multi-scale high-dimensional shallow representation, and then develops attention-based dual-path feature learning modules to extensively exploit diverse palm-vein-specific features. After that, we design visual-consistency and identity-aware loss functions to specially decouple the visual and identity features to reconstruct the adversarial palm-vein images. By doing this, the visual characteristics of palm-vein images can be largely preserved while the identity information is removed in the adversarial palm-vein images, such that high-aggressive adversarial palm-vein samples can be obtained. Extensive white-box and black-box attack experiments conducted on three widely used databases clearly show the effectiveness of the proposed network.</p></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":null,"pages":null},"PeriodicalIF":6.0,"publicationDate":"2024-09-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142252013","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"RBP-DIP: Residual back projection with deep image prior for ill-posed CT reconstruction","authors":"","doi":"10.1016/j.neunet.2024.106740","DOIUrl":"10.1016/j.neunet.2024.106740","url":null,"abstract":"<div><p>The success of deep image prior (DIP) in a number of image processing tasks has motivated their application in image reconstruction problems in computed tomography (CT). In this paper, we introduce a residual back projection technique (RBP) that improves the performance of deep image prior framework in iterative CT reconstruction, especially when the reconstruction problem is highly ill-posed. The RBP-DIP framework uses an untrained U-net in conjunction with a novel residual back projection connection to minimize the objective function while improving reconstruction accuracy. In each iteration, the weights of the untrained U-net are optimized, and the output of the U-net in the current iteration is used to update the input of the U-net in the next iteration through the proposed RBP connection. The introduction of the RBP connection strengthens the regularization effects of the DIP framework in the context of iterative CT reconstruction leading to improvements in accuracy. Our experiments demonstrate that the RBP-DIP framework offers improvements over other state-of-the-art conventional IR methods, as well as pre-trained and untrained models with similar network structures under multiple conditions. These improvements are particularly significant in the few-view and limited-angle CT reconstructions, where the corresponding inverse problems are highly ill-posed and the training data is limited. Furthermore, RBP-DIP has the potential for further improvement. Most existing IR algorithms, pre-trained models, and enhancements applicable to the original DIP algorithm can also be integrated into the RBP-DIP framework.</p></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":null,"pages":null},"PeriodicalIF":6.0,"publicationDate":"2024-09-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142271785","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Joint weight optimization for partial domain adaptation via kernel statistical distance estimation","authors":"","doi":"10.1016/j.neunet.2024.106739","DOIUrl":"10.1016/j.neunet.2024.106739","url":null,"abstract":"<div><p>The goal of Partial Domain Adaptation (PDA) is to transfer a neural network from a source domain (joint source distribution) to a distinct target domain (joint target distribution), where the source label space subsumes the target label space. To address the PDA problem, existing works have proposed to learn the marginal source weights to match the weighted marginal source distribution to the marginal target distribution. However, this is sub-optimal, since the neural network’s target performance is concerned with the joint distribution disparity, not the marginal distribution disparity. In this paper, we propose a Joint Weight Optimization (JWO) approach that optimizes the joint source weights to match the weighted joint source distribution to the joint target distribution in the neural network’s feature space. To measure the joint distribution disparity, we exploit two statistical distances: the distribution-difference-based <span><math><msup><mrow><mi>L</mi></mrow><mrow><mn>2</mn></mrow></msup></math></span>-distance and the distribution-ratio-based <span><math><msup><mrow><mi>χ</mi></mrow><mrow><mn>2</mn></mrow></msup></math></span>-divergence. Since these two distances are unknown in practice, we propose a Kernel Statistical Distance Estimation (KSDE) method to estimate them from the weighted source data and the target data. Our KSDE method explicitly expresses the two estimated statistical distances as functions of the joint source weights. Therefore, we can optimize the joint weights to minimize the estimated distance functions and reduce the joint distribution disparity. Finally, we achieve the PDA goal by training the neural network on the weighted source data. Experiments on several popular datasets are conducted to demonstrate the effectiveness of our approach. Intro video and Pytorch code are available at <span><span>https://github.com/sentaochen/Joint-Weight-Optimation</span><svg><path></path></svg></span>. Interested readers can also visit <span><span>https://github.com/sentaochen</span><svg><path></path></svg></span> for more source codes of the related domain adaptation, multi-source domain adaptation, and domain generalization approaches.</p></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":null,"pages":null},"PeriodicalIF":6.0,"publicationDate":"2024-09-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142243879","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Beyond smoothness: A general optimization framework for graph neural networks with negative Laplacian regularization","authors":"","doi":"10.1016/j.neunet.2024.106704","DOIUrl":"10.1016/j.neunet.2024.106704","url":null,"abstract":"<div><div>Graph Neural Networks (GNNs) have drawn great attention in handling graph-structured data. To characterize the message-passing mechanism of GNNs, recent studies have established a unified framework that models the graph convolution operation as a graph signal denoising problem. While increasing interpretability, this framework often performs poorly on heterophilic graphs and also leads to shallow and fragile GNNs in practice. The key reason is that it encourages feature smoothness, but ignores the high-frequency information of node features. To address this issue, we propose a general framework for GNNs via relaxation of the smoothness regularization. In particular, it employs an information aggregation mechanism to learn the low- and high-frequency components adaptively from data, offering more flexible graph convolution operators compared to the smoothness-promoted framework. Theoretical analyses demonstrate that our framework can capture both low- and high-frequency information of node features, effectively. Experiments on nine benchmark datasets show that our framework achieves the state-of-the-art performance in most cases. Furthermore, it can be used to handle deep models and adversarial attacks.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":null,"pages":null},"PeriodicalIF":6.0,"publicationDate":"2024-09-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142312161","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"GEPAF: A non-monotonic generalized activation function in neural network for improving prediction with diverse data distributions characteristics","authors":"","doi":"10.1016/j.neunet.2024.106738","DOIUrl":"10.1016/j.neunet.2024.106738","url":null,"abstract":"<div><p>The world today has made prescriptive analytics that uses data-driven insights to guide future actions. The distribution of data, however, differs depending on the scenario, making it difficult to interpret and comprehend the data efficiently. Different neural network models are used to solve this, taking inspiration from the complex network architecture in the human brain. The activation function is crucial in introducing non-linearity to process data gradients effectively. Although popular activation functions such as ReLU, Sigmoid, Swish, and Tanh have advantages and disadvantages, they may struggle to adapt to diverse data characteristics. A generalized activation function named the Generalized Exponential Parametric Activation Function (GEPAF) is proposed to address this issue. This function consists of three parameters expressed: <span><math><mi>α</mi></math></span>, which stands for a differencing factor similar to the mean; <span><math><mi>σ</mi></math></span>, which stands for a variance to control distribution spread; and <span><math><mi>p</mi></math></span>, which is a power factor that improves flexibility; all these parameters are present in the exponent. When <span><math><mrow><mi>p</mi><mo>=</mo><mn>2</mn></mrow></math></span>, the activation function resembles a Gaussian function. Initially, this paper describes the mathematical derivation and validation of the properties of this function mathematically and graphically. After this, the GEPAF function is practically implemented in real-world supply chain datasets. One dataset features a small sample size but exhibits high variance, while the other shows significant variance with a moderate amount of data. An LSTM network processes the dataset for sales and profit prediction. The suggested function performs better than popular activation functions when a comparative analysis of the activation function is performed, showing at least 30% improvement in regression evaluation metrics and better loss decay characteristics.</p></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":null,"pages":null},"PeriodicalIF":6.0,"publicationDate":"2024-09-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142271784","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}