Neural NetworksPub Date : 2024-11-05DOI: 10.1016/j.neunet.2024.106857
Suneung Kim, Seong-Whan Lee
{"title":"AmbiBias Contrast: Enhancing debiasing networks via disentangled space from ambiguity-bias clusters","authors":"Suneung Kim, Seong-Whan Lee","doi":"10.1016/j.neunet.2024.106857","DOIUrl":"10.1016/j.neunet.2024.106857","url":null,"abstract":"<div><div>The goal of debiasing in classification tasks is to train models to be less sensitive to correlations between a sample’s target attribution and periodically occurring contextual attributes to achieve accurate classification. A prevalent method involves applying re-weighing techniques to lower the weight of bias-aligned samples that contribute to bias, thereby focusing the training on bias-conflicting samples that deviate from the bias patterns. Our empirical analysis indicates that this approach is effective in datasets where bias-conflicting samples constitute a minority compared to bias-aligned samples, yet its effectiveness diminishes in datasets with similar proportions of both. This ineffectiveness in varied dataset compositions suggests that the traditional method cannot be practical in diverse environments as it overlooks the dynamic nature of dataset-induced biases. To address this issue, we introduce a contrastive approach named “AmbiBias Contrast”, which is robust across various dataset compositions. This method accounts for “ambiguity bias”— the variable nature of bias elements across datasets, which cannot be clearly defined. Given the challenge of defining bias due to the fluctuating compositions of datasets, we designed a method of representation learning that accommodates this ambiguity. Our experiments across a range of and dataset configurations verify the robustness of our method, delivering state-of-the-art performance.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"181 ","pages":"Article 106857"},"PeriodicalIF":6.0,"publicationDate":"2024-11-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142631292","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Neural NetworksPub Date : 2024-11-04DOI: 10.1016/j.neunet.2024.106853
Hanjuan Huang , Hsing-Kuo Pao
{"title":"A unified noise and watermark removal from information bottleneck-based modeling","authors":"Hanjuan Huang , Hsing-Kuo Pao","doi":"10.1016/j.neunet.2024.106853","DOIUrl":"10.1016/j.neunet.2024.106853","url":null,"abstract":"<div><div>Both image denoising and watermark removal aim to restore a clean image from an observed noisy or watermarked one. The past research consists of the non-learning type with limited effectiveness or the learning types with limited interpretability. To address these issues simultaneously, we propose a method to deal with both the image-denoising and watermark removal tasks in a unified approach. The noises and watermarks are both considered to have different nuisance patterns from the original image content, therefore should be detected by robust image analysis. The unified detection method is based on the well-known information bottleneck (IB) theory and the proposed SIB-GAN where image content and nuisance patterns are well separated by a supervised approach. The IB theory guides us to keep the valuable content such as the original image by a controlled compression on the input (the noisy or watermark-included image) and then only the content without the nuisances can go through the network for effective noise or watermark removal. Additionally, we adjust the compression parameter in IB theory to learn a representation that approaches the minimal sufficient representation of the image content. In particular, to deal with the non-blind noises, an appropriate amount of compression can be estimated from the solid theory foundation. Working on the denoising task given the unseen data with blind noises also shows the model’s generalization power. All of the above shows the interpretability of the proposed method. Overall, the proposed method has achieved promising results across three tasks: image denoising, watermark removal, and mixed noise and watermark removal, obtaining resultant images very close to the original image content and owning superior performance to almost all state-of-the-art approaches that deal with the same tasks.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"181 ","pages":"Article 106853"},"PeriodicalIF":6.0,"publicationDate":"2024-11-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142644984","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Neural NetworksPub Date : 2024-11-04DOI: 10.1016/j.neunet.2024.106845
Shubham Pateria, Budhitama Subagdja, Ah-Hwee Tan
{"title":"FedART: A neural model integrating federated learning and adaptive resonance theory","authors":"Shubham Pateria, Budhitama Subagdja, Ah-Hwee Tan","doi":"10.1016/j.neunet.2024.106845","DOIUrl":"10.1016/j.neunet.2024.106845","url":null,"abstract":"<div><div>Federated Learning (FL) has emerged as a promising paradigm for collaborative model training across distributed clients while preserving data privacy. However, prevailing FL approaches aggregate the clients’ local models into a global model through multi-round iterative parameter averaging. This leads to the undesirable bias of the aggregated model towards certain clients in the presence of heterogeneous data distributions among the clients. Moreover, such approaches are restricted to supervised classification tasks and do not support unsupervised clustering. To address these limitations, we propose a novel one-shot FL approach called Federated Adaptive Resonance Theory (FedART) which leverages self-organizing Adaptive Resonance Theory (ART) models to learn category codes, where each code represents a cluster of similar data samples. In FedART, the clients learn to associate their private data with various local category codes. Under heterogeneity, the local codes across different clients represent heterogeneous data. In turn, a global model takes these local codes as inputs and aggregates them into global category codes, wherein heterogeneous client data is indirectly represented by distinctly encoded global codes, in contrast to the averaging out of parameters in the existing approaches. This enables the learned global model to handle heterogeneous data. In addition, FedART employs a universal learning mechanism to support both federated classification and clustering tasks. Our experiments conducted on various federated classification and clustering tasks show that FedART consistently outperforms state-of-the-art FL methods on data with heterogeneous distribution across clients.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"181 ","pages":"Article 106845"},"PeriodicalIF":6.0,"publicationDate":"2024-11-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142631299","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Neural NetworksPub Date : 2024-11-04DOI: 10.1016/j.neunet.2024.106873
Qian Cui , Jinde Cao , Mahmoud Abdel-Aty , Ardak Kashkynbayev
{"title":"Global practical finite-time synchronization of disturbed inertial neural networks by delayed impulsive control","authors":"Qian Cui , Jinde Cao , Mahmoud Abdel-Aty , Ardak Kashkynbayev","doi":"10.1016/j.neunet.2024.106873","DOIUrl":"10.1016/j.neunet.2024.106873","url":null,"abstract":"<div><div>This paper delves into the practical finite-time synchronization (FTS) problem for inertial neural networks (INNs) with external disturbances. Firstly, based on Lyapunov theory, the local practical FTS of INNs with bounded external disturbances can be realized by effective finite time control. Then, building upon the local results, we extend the synchronization to a global practical level under delayed impulsive control. By designing appropriate hybrid controllers, the global practical FTS criteria of disturbed INNs are obtained and the corresponding settling time is estimated. In addition, for impulsive control, the maximum impulsive interval is used to describe the frequency at which the impulses occur. We optimize the maximum impulsive interval, aiming to minimize impulses occurrence, which directly translates to reduced control costs. Moreover, by comparing the global FTS results for INNs without external disturbances, it can be found that the existence of perturbations necessitates either higher impulsive intensity or denser impulses to maintain networks synchronization. Two examples are shown to demonstrate the reasonableness of designed hybrid controllers.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"181 ","pages":"Article 106873"},"PeriodicalIF":6.0,"publicationDate":"2024-11-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142631304","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Neural NetworksPub Date : 2024-11-04DOI: 10.1016/j.neunet.2024.106854
Huilan Luo , Weixia Hu , Yixiao Wei , Jianlong He , Minghao Yu
{"title":"HirMTL: Hierarchical Multi-Task Learning for dense scene understanding","authors":"Huilan Luo , Weixia Hu , Yixiao Wei , Jianlong He , Minghao Yu","doi":"10.1016/j.neunet.2024.106854","DOIUrl":"10.1016/j.neunet.2024.106854","url":null,"abstract":"<div><div>In the realm of artificial intelligence, simultaneous multi-task learning is crucial, particularly for dense scene understanding. To address this, we introduce HirMTL, a novel hierarchical multi-task learning framework designed to enhance dense scene analysis. HirMTL is adept at facilitating interaction at the scale level, ensuring task-adaptive multi-scale feature fusion, and fostering task-level feature interchange. It leverages the inherent correlations between tasks to create a synergistic learning environment. Initially, HirMTL enables concurrent sharing and fine-tuning of features at the single-scale level. This is further extended by the Task-Adaptive Fusion module (TAF), which intelligently blends features across scales, specifically attuned to each task’s unique requirements. Complementing this, the Asymmetric Information Comparison Module (AICM) skillfully differentiates and processes both shared and unique features, significantly refining task-specific performance with enhanced accuracy. Our extensive experiments on various dense prediction tasks validate HirMTL’s exceptional capabilities, showcasing its superiority over existing multi-task learning models and underscoring the benefits of its hierarchical approach.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"181 ","pages":"Article 106854"},"PeriodicalIF":6.0,"publicationDate":"2024-11-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142631413","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Neural NetworksPub Date : 2024-11-02DOI: 10.1016/j.neunet.2024.106847
Davide Borra , Elisa Magosso , Mirco Ravanelli
{"title":"A protocol for trustworthy EEG decoding with neural networks","authors":"Davide Borra , Elisa Magosso , Mirco Ravanelli","doi":"10.1016/j.neunet.2024.106847","DOIUrl":"10.1016/j.neunet.2024.106847","url":null,"abstract":"<div><div>Deep learning solutions have rapidly emerged for EEG decoding, achieving state-of-the-art performance on a variety of decoding tasks. Despite their high performance, existing solutions do not fully address the challenge posed by the introduction of many hyperparameters, defining data pre-processing, network architecture, network training, and data augmentation. Automatic hyperparameter search is rarely performed and limited to network-related hyperparameters. Moreover, pipelines are highly sensitive to performance fluctuations due to random initialization, hindering their reliability. Here, we design a comprehensive protocol for EEG decoding that explores the hyperparameters characterizing the entire pipeline and that includes multi-seed initialization for providing robust performance estimates. Our protocol is validated on 9 datasets about motor imagery, P300, SSVEP, including 204 participants and 26 recording sessions, and on different deep learning models. We accompany our protocol with extensive experiments on the main aspects influencing it, such as the number of participants used for hyperparameter search, the split into sequential simpler searches (multi-step search), the use of informed vs. non-informed search algorithms, and the number of random seeds for obtaining stable performance. The best protocol included 2-step hyperparameter search via an informed search algorithm, with the final training and evaluation performed using 10 random initializations. The optimal trade-off between performance and computational time was achieved by using a subset of 3–5 participants for hyperparameter search. Our protocol consistently outperformed baseline state-of-the-art pipelines, widely across datasets and models, and could represent a standard approach for neuroscientists for decoding EEG in a trustworthy and reliable way.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"182 ","pages":"Article 106847"},"PeriodicalIF":6.0,"publicationDate":"2024-11-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142645080","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Neural NetworksPub Date : 2024-11-02DOI: 10.1016/j.neunet.2024.106852
Tong Li , Chenjia Bai , Kang Xu , Chen Chu , Peican Zhu , Zhen Wang
{"title":"Skill matters: Dynamic skill learning for multi-agent cooperative reinforcement learning","authors":"Tong Li , Chenjia Bai , Kang Xu , Chen Chu , Peican Zhu , Zhen Wang","doi":"10.1016/j.neunet.2024.106852","DOIUrl":"10.1016/j.neunet.2024.106852","url":null,"abstract":"<div><div>With the popularization of intelligence, the necessity of cooperation between intelligent machines makes the research of collaborative multi-agent reinforcement learning (MARL) more extensive. Existing approaches typically address this challenge through task decomposition of the environment or role classification of agents. However, these studies may rely on the sharing of parameters between agents, resulting in the homogeneity of agent behavior, which is not effective for complex tasks. Or training that relies on external rewards is difficult to adapt to scenarios with sparse rewards. Based on the above challenges, in this paper we propose a novel dynamic skill learning (DSL) framework for agents to learn more diverse abilities motivated by internal rewards. Specifically, the DSL has two components: (i) Dynamic skill discovery, which encourages the production of meaningful skills by exploring the environment in an unsupervised manner, using the inner product between a skill vector and a trajectory representation to generate intrinsic rewards. Meanwhile, the Lipschitz constraint of the state representation function is used to ensure the proper trajectory of the learned skills. (ii) Dynamic skill assignment, which utilizes a policy controller to assign skills to each agent based on its different trajectory latent variables. In addition, in order to avoid training instability caused by frequent changes in skill selection, we introduce a regularization term to limit skill switching between adjacent time steps. We thoroughly tested the DSL approach on two challenging benchmarks, StarCraft II and Google Research Football. Experimental results show that compared with strong benchmarks such as QMIX and RODE, DSL effectively improves performance and is more adaptable to difficult collaborative scenarios.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"181 ","pages":"Article 106852"},"PeriodicalIF":6.0,"publicationDate":"2024-11-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142631417","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Neural NetworksPub Date : 2024-11-02DOI: 10.1016/j.neunet.2024.106851
Ziyu Wang, Yiming Du, Yao Wang, Rui Ning, Lusi Li
{"title":"Deep Incomplete Multi-view Clustering via Multi-level Imputation and Contrastive Alignment","authors":"Ziyu Wang, Yiming Du, Yao Wang, Rui Ning, Lusi Li","doi":"10.1016/j.neunet.2024.106851","DOIUrl":"10.1016/j.neunet.2024.106851","url":null,"abstract":"<div><div>Deep incomplete multi-view clustering (DIMVC) aims to enhance clustering performance by capturing consistent information from incomplete multiple views using deep models. Most existing DIMVC methods typically employ imputation-based strategies to handle missing views before clustering. However, they often assume complete data availability across all views, overlook potential low-quality views, and perform imputation at a single data level, leading to challenges in accurately inferring missing data. To address these issues, we propose a novel imputation-based approach called Multi-level Imputation and Contrastive Alignment (MICA) to simultaneously improve imputation quality and boost clustering performance. Specifically, MICA employs an individual deep model for each view, which unifies view feature learning and cluster assignment prediction. It leverages the learned features from available instances to construct an adaptive cross-view graph for reliable view selection. Guided by these reliable views, MICA performs multi-level (feature-level, data-level, and reconstruction-level) imputation to preserve topological structures across levels and ensure accurate missing feature inference. The complete features are then used for discriminative cluster assignment learning. Additionally, an instance- and cluster-level contrastive alignment is conducted on the cluster assignments to further enhance semantic consistency across views. Experimental results show the effectiveness and superior performance of the proposed MICA method.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"181 ","pages":"Article 106851"},"PeriodicalIF":6.0,"publicationDate":"2024-11-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142607185","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Neural NetworksPub Date : 2024-11-02DOI: 10.1016/j.neunet.2024.106859
Chenyang Zhu , Lanlan Zhang , Weibin Luo , Guangqi Jiang , Qian Wang
{"title":"Tensorial multiview low-rank high-order graph learning for context-enhanced domain adaptation","authors":"Chenyang Zhu , Lanlan Zhang , Weibin Luo , Guangqi Jiang , Qian Wang","doi":"10.1016/j.neunet.2024.106859","DOIUrl":"10.1016/j.neunet.2024.106859","url":null,"abstract":"<div><div>Unsupervised Domain Adaptation (UDA) is a machine learning technique that facilitates knowledge transfer from a labeled source domain to an unlabeled target domain, addressing distributional discrepancies between these domains. Existing UDA methods often fail to effectively capture and utilize contextual relationships within the target domain. This research introduces a novel framework called Tensorial Multiview Low-Rank High-Order Graph Learning (MLRGL), which addresses these challenges by learning high-order graphs constrained by low-rank tensors to uncover contextual relations. The proposed framework ensures prediction consistency between randomly masked target images and their pseudo-labels by leveraging spatial context to generate multiview domain-invariant features through various augmented masking techniques. A high-order graph is constructed by combining Laplacian graphs to propagate these multiview features. Low-rank constraints are applied along both horizontal and vertical dimensions to better uncover inter-view and inter-class correlations among multiview features. This high-order graph is used to create an affinity matrix, mapping multiview features into a unified subspace. Prototype vectors and unsupervised clustering are then employed to calculate conditional probabilities for UDA tasks. We evaluated our approach using three different backbones across three benchmark datasets. The results demonstrate that the MLRGL framework outperforms current state-of-the-art methods in various UDA tasks. Additionally, our framework exhibits robustness to hyperparameter variations and demonstrates that multiview approaches outperform single-view solutions.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"181 ","pages":"Article 106859"},"PeriodicalIF":6.0,"publicationDate":"2024-11-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142607221","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Contrastive fine-grained domain adaptation network for EEG-based vigilance estimation.","authors":"Kangning Wang, Wei Wei, Weibo Yi, Shuang Qiu, Huiguang He, Minpeng Xu, Dong Ming","doi":"10.1016/j.neunet.2024.106617","DOIUrl":"10.1016/j.neunet.2024.106617","url":null,"abstract":"<p><p>Vigilance state is crucial for the effective performance of users in brain-computer interface (BCI) systems. Most vigilance estimation methods rely on a large amount of labeled data to train a satisfactory model for the specific subject, which limits the practical application of the methods. This study aimed to build a reliable vigilance estimation method using a small amount of unlabeled calibration data. We conducted a vigilance experiment in the designed BCI-based cursor-control task. Electroencephalogram (EEG) signals of eighteen participants were recorded in two sessions on two different days. And, we proposed a contrastive fine-grained domain adaptation network (CFGDAN) for vigilance estimation. Here, an adaptive graph convolution network (GCN) was built to project the EEG data of different domains into a common space. The fine-grained feature alignment mechanism was designed to weight and align the feature distributions across domains at the EEG channel level, and the contrastive information preservation module was developed to preserve the useful target-specific information during the feature alignment. The experimental results show that the proposed CFGDAN outperforms the compared methods in our BCI vigilance dataset and SEED-VIG dataset. Moreover, the visualization results demonstrate the efficacy of the designed feature alignment mechanisms. These results indicate the effectiveness of our method for vigilance estimation. Our study is helpful for reducing calibration efforts and promoting the practical application potential of vigilance estimation methods.</p>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"179 ","pages":"106617"},"PeriodicalIF":6.0,"publicationDate":"2024-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142057086","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}