IEEE transactions on neural networks and learning systems最新文献

{"title":"Tunnel Prescribed Control of Nonlinear Systems With Unknown Control Directions.","authors":"Ruihang Ji, Dongyu Li, Shuzhi Sam Ge, Haizhou Li","doi":"10.1109/TNNLS.2023.3322161","DOIUrl":"10.1109/TNNLS.2023.3322161","url":null,"abstract":"<p><p>This article solves the entry capture problem (ECP) such that for any initial tracking error, it can be regulated into the prescribed performance constraints within a user-given time. The challenge lies in how to remove the initial condition limitation and to handle the ECP for nonlinear systems under unknown control directions and asymmetric performance constraints. For better tracking performance, we propose a unified tunnel prescribed performance (TPP) providing strict and tight allowable set. With the aid of a scaling function, error self-tuning functions (ESFs) are then developed to make the control scheme suitable to any initial condition (including the initial constraint violation), where the initial values of ESFs always satisfy performance constraints. In lieu of the Nussbaum technique, an orientation function is introduced to deal with unknown control directions while such way is capable of reducing the control peaking problem. Using ESFs, together with TPP and an orientation function, the resulted tunnel prescribed control (TPC) leads to a solution for the underlying ECP, which also exhibits a low complexity level since no command filters or dynamic surface control is required. Finally, simulation results are provided to further demonstrate these theoretical findings.</p>","PeriodicalId":13303,"journal":{"name":"IEEE transactions on neural networks and learning systems","volume":"PP ","pages":""},"PeriodicalIF":10.4,"publicationDate":"2023-10-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41234971","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Convergence Analysis of Online Gradient Method for High-Order Neural Networks and Their Sparse Optimization.","authors":"Qinwei Fan, Qian Kang, Jacek M Zurada, Tingwen Huang, Dongpo Xu","doi":"10.1109/TNNLS.2023.3319989","DOIUrl":"10.1109/TNNLS.2023.3319989","url":null,"abstract":"<p><p>In this article, we investigate the boundedness and convergence of the online gradient method with the smoothing group L_1/2 regularization for the sigma-pi-sigma neural network (SPSNN). This enhances the sparseness of the network and improves its generalization ability. For the original group L_1/2 regularization, the error function is nonconvex and nonsmooth, which can cause oscillation of the error function. To ameliorate this drawback, we propose a simple and effective smoothing technique, which can effectively eliminate the deficiency of the original group L_1/2 regularization. The group L_1/2 regularization effectively optimizes the network structure from two aspects redundant hidden nodes tending to zero and redundant weights of surviving hidden nodes in the network tending to zero. This article shows the strong and weak convergence results for the proposed method and proves the boundedness of weights. Experiment results clearly demonstrate the capability of the proposed method and the effectiveness of redundancy control. The simulation results are observed to support the theoretical results.</p>","PeriodicalId":13303,"journal":{"name":"IEEE transactions on neural networks and learning systems","volume":"PP ","pages":""},"PeriodicalIF":10.4,"publicationDate":"2023-10-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41234963","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Toward Robust Discriminative Projections Learning Against Adversarial Patch Attacks.","authors":"Zheng Wang, Feiping Nie, Hua Wang, Heng Huang, Fei Wang","doi":"10.1109/TNNLS.2023.3321606","DOIUrl":"10.1109/TNNLS.2023.3321606","url":null,"abstract":"<p><p>As one of the most popular supervised dimensionality reduction methods, linear discriminant analysis (LDA) has been widely studied in machine learning community and applied to many scientific applications. Traditional LDA minimizes the ratio of squared l₂ norms, which is vulnerable to the adversarial examples. In recent studies, many l₁ -norm-based robust dimensionality reduction methods are proposed to improve the robustness of model. However, due to the difficulty of l₁ -norm ratio optimization and weakness on defending a large number of adversarial examples, so far, scarce works have been proposed to utilize sparsity-inducing norms for LDA objective. In this article, we propose a novel robust discriminative projections learning (rDPL) method based on the l_1,2 -norm trace-ratio minimization optimization algorithm. Minimizing the l_1,2 -norm ratio problem directly is a much more challenging problem than the traditional methods, and there is no existing optimization algorithm to solve such nonsmooth terms ratio problem. We derive a new efficient algorithm to solve this challenging problem and provide a theoretical analysis on the convergence of our algorithm. The proposed algorithm is easy to implement and converges fast in practice. Extensive experiments on both synthetic data and several real benchmark datasets show the effectiveness of the proposed method on defending the adversarial patch attack by comparison with many state-of-the-art robust dimensionality reduction methods.</p>","PeriodicalId":13303,"journal":{"name":"IEEE transactions on neural networks and learning systems","volume":"PP ","pages":""},"PeriodicalIF":10.4,"publicationDate":"2023-10-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41234970","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Explainable Classification of Benign-Malignant Pulmonary Nodules With Neural Networks and Information Bottleneck.","authors":"Haixing Zhu, Weipeng Liu, Zhifan Gao, Heye Zhang","doi":"10.1109/TNNLS.2023.3303395","DOIUrl":"10.1109/TNNLS.2023.3303395","url":null,"abstract":"<p><p>Computerized tomography (CT) is a clinically primary technique to differentiate benign-malignant pulmonary nodules for lung cancer diagnosis. Early classification of pulmonary nodules is essential to slow down the degenerative process and reduce mortality. The interactive paradigm assisted by neural networks is considered to be an effective means for early lung cancer screening in large populations. However, some inherent characteristics of pulmonary nodules in high-resolution CT images, e.g., diverse shapes and sparse distribution over the lung fields, have been inducing inaccurate results. On the other hand, most existing methods with neural networks are dissatisfactory from a lack of transparency. In order to overcome these obstacles, a united framework is proposed, including the classification and feature visualization stages, to learn distinctive features and provide visual results. Specifically, a bilateral scheme is employed to synchronously extract and aggregate global-local features in the classification stage, where the global branch is constructed to perceive deep-level features and the local branch is built to focus on the refined details. Furthermore, an encoder is built to generate some features, and a decoder is constructed to simulate decision behavior, followed by the information bottleneck viewpoint to optimize the objective. Extensive experiments are performed to evaluate our framework on two publicly available datasets, namely, 1) the Lung Image Database Consortium and Image Database Resource Initiative (LIDC-IDRI) and 2) the Lung and Colon Histopathological Image Dataset (LC25000). For instance, our framework achieves 92.98% accuracy and presents additional visualizations on the LIDC. The experiment results show that our framework can obtain outstanding performance and is effective to facilitate explainability. It also demonstrates that this united framework is a serviceable tool and further has the scalability to be introduced into clinical research.</p>","PeriodicalId":13303,"journal":{"name":"IEEE transactions on neural networks and learning systems","volume":"PP ","pages":""},"PeriodicalIF":10.4,"publicationDate":"2023-10-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41234966","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Bayesian Hierarchical Graph Neural Networks With Uncertainty Feedback for Trustworthy Fault Diagnosis of Industrial Processes.","authors":"Dongyue Chen, Zongxia Xie, Ruonan Liu, Wenlong Yu, Qinghua Hu, Xianling Li, Steven X Ding","doi":"10.1109/TNNLS.2023.3319468","DOIUrl":"10.1109/TNNLS.2023.3319468","url":null,"abstract":"<p><p>Deep learning (DL) methods have been widely applied to intelligent fault diagnosis of industrial processes and achieved state-of-the-art performance. However, fault diagnosis with point estimate may provide untrustworthy decisions. Recently, Bayesian inference shows to be a promising approach to trustworthy fault diagnosis by quantifying the uncertainty of the decisions with a DL model. The uncertainty information is not involved in the training process, which does not help the learning of highly uncertain samples and has little effect on improving the fault diagnosis performance. To address this challenge, we propose a Bayesian hierarchical graph neural network (BHGNN) with an uncertainty feedback mechanism, which formulates a trustworthy fault diagnosis on the Bayesian DL (BDL) framework. Specifically, BHGNN captures the epistemic uncertainty and aleatoric uncertainty via a variational dropout approach and utilizes the uncertainty information of each sample to adjust the strength of the temporal consistency (TC) constraint for robust feature learning. Meanwhile, the BHGNN method models the process data as a hierarchical graph (HG) by leveraging the interaction-aware module and physical topology knowledge of the industrial process, which integrates data with domain knowledge to learn fault representation. Moreover, the experiments on a three-phase flow facility (TFF) and secure water treatment (SWaT) show superior and competitive performance in fault diagnosis and verify the trustworthiness of the proposed method.</p>","PeriodicalId":13303,"journal":{"name":"IEEE transactions on neural networks and learning systems","volume":"PP ","pages":""},"PeriodicalIF":10.4,"publicationDate":"2023-10-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41234961","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A Generalized Nesterov-Accelerated Second-Order Latent Factor Model for High-Dimensional and Incomplete Data.","authors":"Weiling Li, Renfang Wang, Xin Luo","doi":"10.1109/TNNLS.2023.3321915","DOIUrl":"10.1109/TNNLS.2023.3321915","url":null,"abstract":"<p><p>High-dimensional and incomplete (HDI) data are frequently encountered in big date-related applications for describing restricted observed interactions among large node sets. How to perform accurate and efficient representation learning on such HDI data is a hot yet thorny issue. A latent factor (LF) model has proven to be efficient in addressing it. However, the objective function of an LF model is nonconvex. Commonly adopted first-order methods cannot approach its second-order stationary point, thereby resulting in accuracy loss. On the other hand, traditional second-order methods are impractical for LF models since they suffer from high computational costs due to the required operations on the objective's huge Hessian matrix. In order to address this issue, this study proposes a generalized Nesterov-accelerated second-order LF (GNSLF) model that integrates twofold conceptions: 1) acquiring proper second-order step efficiently by adopting a Hessian-vector algorithm and 2) embedding the second-order step into a generalized Nesterov's acceleration (GNA) method for speeding up its linear search process. The analysis focuses on the local convergence for GNSLF's nonconvex cost function instead of the global convergence has been taken; its local convergence properties have been provided with theoretical proofs. Experimental results on six HDI data cases demonstrate that GNSLF performs better than state-of-the-art LF models in accuracy for missing data estimation with high efficiency, i.e., a second-order model can be accelerated by incorporating GNA without accuracy loss.</p>","PeriodicalId":13303,"journal":{"name":"IEEE transactions on neural networks and learning systems","volume":"PP ","pages":""},"PeriodicalIF":10.4,"publicationDate":"2023-10-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41199547","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Hybrid Network Using Dynamic Graph Convolution and Temporal Self-Attention for EEG-Based Emotion Recognition.","authors":"Cheng Cheng, Zikang Yu, Yong Zhang, Lin Feng","doi":"10.1109/TNNLS.2023.3319315","DOIUrl":"10.1109/TNNLS.2023.3319315","url":null,"abstract":"<p><p>The electroencephalogram (EEG) signal has become a highly effective decoding target for emotion recognition and has garnered significant attention from researchers. Its spatial topological and time-dependent characteristics make it crucial to explore both spatial information and temporal information for accurate emotion recognition. However, existing studies often focus on either spatial or temporal aspects of EEG signals, neglecting the joint consideration of both perspectives. To this end, this article proposes a hybrid network consisting of a dynamic graph convolution (DGC) module and temporal self-attention representation (TSAR) module, which concurrently incorporates the representative knowledge of spatial topology and temporal context into the EEG emotion recognition task. Specifically, the DGC module is designed to capture the spatial functional relationships within the brain by dynamically updating the adjacency matrix during the model training process. Simultaneously, the TSAR module is introduced to emphasize more valuable time segments and extract global temporal features from EEG signals. To fully exploit the interactivity between spatial and temporal information, the hierarchical cross-attention fusion (H-CAF) module is incorporated to fuse the complementary information from spatial and temporal features. Extensive experimental results on the DEAP, SEED, and SEED-IV datasets demonstrate that the proposed method outperforms other state-of-the-art methods.</p>","PeriodicalId":13303,"journal":{"name":"IEEE transactions on neural networks and learning systems","volume":"PP ","pages":""},"PeriodicalIF":10.4,"publicationDate":"2023-10-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41199554","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Diffusion Source Inference for Large-Scale Complex Networks Based on Network Percolation.","authors":"Yang Liu, Xiaoqi Wang, Xi Wang, Zhen Wang, Jurgen Kurths","doi":"10.1109/TNNLS.2023.3321767","DOIUrl":"10.1109/TNNLS.2023.3321767","url":null,"abstract":"<p><p>This article studies the diffusion-source-inference (DSI) problem, whose solution plays an important role in real-world scenarios such as combating misinformation and controlling diffusions of information or disease. The main task of the DSI problem is to optimize an estimator, such that the real source can be more precisely targeted. In this article, we assume that the state of a number of nodes, called observer set, in a network could be investigated if necessary, and study what configuration of those nodes could facilitate a better solution for the DSI problem. In particular, we find that the conventional error distance metric cannot precisely evaluate the effectiveness of varied DSI approaches in heterogeneous networks, and thus propose a novel and more general measurement, the candidate set, that is formulated to contain the diffusion source for sure. We propose the percolation-based evolutionary framework (PrEF) to optimize the observer set such that the candidate set can be minimized. Hence, one could further conduct more intensive investigation or search on only a few nodes to target the source. To achieve that, we first theoretically show that the size of the candidate set is bounded by the size of the largest component cover, and demonstrate that there are some similarities between the DSI problem and the network immunization problem. We find that, given the associated direction information of the diffusion is known on observers, the minimization of the candidate set is equivalent to the minimization of the order parameter if we view the observer set as the removal node set. Hence, PrEF is developed based on the network percolation and evolutionary algorithm. The effectiveness of the proposed method is validated on both synthetic and empirical networks in regard to varied circumstances. Our results show that the developed approach could achieve much smaller candidate sets compared to the state of the art in almost all cases, e.g., it is better in 26 out of 27 empirical networks and 155 out of 162 cases regarding the critical threshold. Meanwhile, our approach is also more stable, i.e., it works well irrespective of varied infection probabilities, diffusion models, and underlying networks. More importantly, we provide a framework for the analysis of the DSI problem in large-scale networks.</p>","PeriodicalId":13303,"journal":{"name":"IEEE transactions on neural networks and learning systems","volume":"PP ","pages":""},"PeriodicalIF":10.4,"publicationDate":"2023-10-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41199552","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Physical Adversarial Attacks for Surveillance: A Survey.","authors":"Kien Nguyen, Tharindu Fernando, Clinton Fookes, Sridha Sridharan","doi":"10.1109/TNNLS.2023.3321432","DOIUrl":"10.1109/TNNLS.2023.3321432","url":null,"abstract":"<p><p>Modern automated surveillance techniques are heavily reliant on deep learning methods. Despite the superior performance, these learning systems are inherently vulnerable to adversarial attacks-maliciously crafted inputs that are designed to mislead, or trick, models into making incorrect predictions. An adversary can physically change their appearance by wearing adversarial t-shirts, glasses, or hats or by specific behavior, to potentially avoid various forms of detection, tracking, and recognition of surveillance systems; and obtain unauthorized access to secure properties and assets. This poses a severe threat to the security and safety of modern surveillance systems. This article reviews recent attempts and findings in learning and designing physical adversarial attacks for surveillance applications. In particular, we propose a framework to analyze physical adversarial attacks and provide a comprehensive survey of physical adversarial attacks on four key surveillance tasks: detection, identification, tracking, and action recognition under this framework. Furthermore, we review and analyze strategies to defend against physical adversarial attacks and the methods for evaluating the strengths of the defense. The insights in this article present an important step in building resilience within surveillance systems to physical adversarial attacks.</p>","PeriodicalId":13303,"journal":{"name":"IEEE transactions on neural networks and learning systems","volume":"PP ","pages":""},"PeriodicalIF":10.4,"publicationDate":"2023-10-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41199560","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Manipulating Identical Filter Redundancy for Efficient Pruning on Deep and Complicated CNN","authors":"Tianxiang Hao;Xiaohan Ding;Jungong Han;Yuchen Guo;Guiguang Ding","doi":"10.1109/TNNLS.2023.3298263","DOIUrl":"10.1109/TNNLS.2023.3298263","url":null,"abstract":"The existence of redundancy in convolutional neural networks (CNNs) enables us to remove some filters/channels with acceptable performance drops. However, the training objective of CNNs usually tends to minimize an accuracy-related loss function without any attention paid to the redundancy, making the redundancy distribute randomly on all the filters, such that removing any of them may trigger information loss and accuracy drop, necessitating a fine-tuning step for recovery. In this article, we propose to manipulate the redundancy during training to facilitate network pruning. To this end, we propose a novel centripetal SGD (C-SGD) to make some filters identical, resulting in ideal redundancy patterns, as such filters become purely redundant due to their duplicates, hence removing them does not harm the network. As shown on CIFAR and ImageNet, C-SGD delivers better performance because the redundancy is better organized, compared to the existing methods. The efficiency also characterizes C-SGD because it is as fast as regular SGD, requires no fine-tuning, and can be conducted simultaneously on all the layers even in very deep CNNs. Besides, C-SGD can improve the accuracy of CNNs by first training a model with the same architecture but wider layers and then squeezing it into the original width.","PeriodicalId":13303,"journal":{"name":"IEEE transactions on neural networks and learning systems","volume":"35 11","pages":"16831-16844"},"PeriodicalIF":10.2,"publicationDate":"2023-10-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41199555","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}