Neurocomputing最新文献_第3页

Mitigating gradient conflicts via expert squads in multi-task learning 通过多任务学习中的专家小组缓解梯度冲突

IF 5.5 2区计算机科学

Neurocomputing Pub Date : 2024-11-08 DOI: 10.1016/j.neucom.2024.128832

Jie Chen, Meng Joo Er

{"title":"Mitigating gradient conflicts via expert squads in multi-task learning","authors":"Jie Chen, Meng Joo Er","doi":"10.1016/j.neucom.2024.128832","DOIUrl":"10.1016/j.neucom.2024.128832","url":null,"abstract":"<div><div>The foundation of multi-task learning lies in the collaboration and interaction among tasks. However, in numerous real-world scenarios, certain tasks usually necessitate distinct, specialized knowledge. The mixing of these different task-specific knowledge often results in gradient conflicts during the optimization process, posing a significant challenge in the design of effective multi-task learning systems. This study proposes a straightforward yet effective multi-task learning framework that employs groups of expert networks to decouple the learning of task-specific knowledge and mitigate such gradient conflicts. Specifically, this approach partitions the feature channels into task-specific and shared components. The task-specific subsets are processed by dedicated experts to distill specialized knowledge. The shared features are captured by a point-wise aggregation layer from the whole outputs of all experts, demonstrating superior performance in capturing inter-task interactions. By considering both task-specific knowledge and shared features, the proposed approach exhibits superior performance in multi-task learning. Extensive experiments conducted on the PASCAL-Context and NYUD-v2 datasets have demonstrated the superiority of the proposed approach compared to other state-of-the-art methods. Furthermore, a benchmark dataset for multi-task learning in underwater scenarios has been developed, encompassing object detection and underwater image enhancement tasks. Comprehensive experiments on this dataset consistently validate the effectiveness of the proposed multi-task learning strategy. The source code is available at <span><span>https://github.com/chenjie04/Multi-Task-Learning-PyTorch</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"614 ","pages":"Article 128832"},"PeriodicalIF":5.5,"publicationDate":"2024-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142660671","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Secure bipartite consensus of leader–follower multi-agent systems under denial-of-service attacks via observer-based dynamic event-triggered control 通过基于观察者的动态事件触发控制，实现拒绝服务攻击下领导者-追随者多代理系统的安全双方共识

IF 5.5 2区计算机科学

Neurocomputing Pub Date : 2024-11-08 DOI: 10.1016/j.neucom.2024.128817

Haichuan Xu , Fanglai Zhu , Xufeng Ling

引用次数: 0

Interpretable neural network classification model using first-order logic rules 使用一阶逻辑规则的可解释神经网络分类模型

IF 5.5 2区计算机科学

Neurocomputing Pub Date : 2024-11-08 DOI: 10.1016/j.neucom.2024.128840

Haiming Tuo, Zuqiang Meng, Zihao Shi, Daosheng Zhang

{"title":"Interpretable neural network classification model using first-order logic rules","authors":"Haiming Tuo, Zuqiang Meng, Zihao Shi, Daosheng Zhang","doi":"10.1016/j.neucom.2024.128840","DOIUrl":"10.1016/j.neucom.2024.128840","url":null,"abstract":"<div><div>Over the past decade, the field of neural networks has made significant strides, particularly in deep learning. However, their limited interpretability has constrained their application in certain critical domains, drawing widespread criticism. Researchers have proposed various methods for explaining neural networks to address this challenge. This paper focuses on rule-based explanations for neural network classification problems. We propose IRCnet, a scalable classification model based on first-order logic rules. IRCnet consists of layers for learning conjunction and disjunction rules, utilizing binary logic activation functions to enhance interpretability. The model is initially trained using a continuous-weight version, which is later binarized to produce a discrete-weight version. During training, we innovatively employed gradient approximation method to handle the non-differentiable weight binarization function, thereby enabling the training of split matrices used for binarization. Finally, Conjunctive Normal Form (CNF) or Disjunctive Normal Form (DNF) rules are extracted from the model’s discrete-weight version. Experimental results indicate that our model achieves the highest or near-highest performance across various classification metrics in multiple structured datasets while demonstrating significant scalability. It effectively balances classification accuracy with the complexity of the generated rules.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"614 ","pages":"Article 128840"},"PeriodicalIF":5.5,"publicationDate":"2024-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142660669","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Event-triggered explorized IRL-based decentralized fault-tolerant guaranteed cost control for interconnected systems against actuator failures 基于事件触发的 IRL 分散容错保证成本控制，用于互联系统，防止执行器故障

IF 5.5 2区计算机科学

Neurocomputing Pub Date : 2024-11-08 DOI: 10.1016/j.neucom.2024.128837

Yuling Liang , Yanhong Luo , Hanguang Su , Xiaoling Zhang , Hongbin Chang , Jun Zhang

{"title":"Event-triggered explorized IRL-based decentralized fault-tolerant guaranteed cost control for interconnected systems against actuator failures","authors":"Yuling Liang , Yanhong Luo , Hanguang Su , Xiaoling Zhang , Hongbin Chang , Jun Zhang","doi":"10.1016/j.neucom.2024.128837","DOIUrl":"10.1016/j.neucom.2024.128837","url":null,"abstract":"<div><div>This paper presents a novel data-based decentralized guaranteed cost (DGC) fault tolerant control (FTC) scheme for the large-scale systems subject to actuator faults and mismatched interconnection. First, the FTC issues of interconnected systems are converted into a series of near optimal event-triggered control (ETC) methods for isolated subsystems via constructing a modified performance index function of each subsystem. By means of adaptive dynamic programming (ADP) algorithm, the upper bound of performance index function of large-scale systems can be obtained by solving the Hamilton-Jacobi-Bellman (HJB) equation of each auxiliary subsystem. Second, according to the proposed ADP-based decentralized approach and utilizing the event-based synchronous integral reinforcement learning (IRL) algorithm, a model-free guaranteed cost (GC) FTC approach is developed for interconnected large-scale system which can relax the restriction on the condition that system functions must be known. Further, the ultimate uniformly bounded (UUB) stability of auxiliary subsystems can be proved according to the Lyapunov principle. Finally, the effectiveness of the proposed control method is verified by presenting the simulation results.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"615 ","pages":"Article 128837"},"PeriodicalIF":5.5,"publicationDate":"2024-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142652695","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Long–Short Observation-driven Prediction Network for pedestrian crossing intention prediction with momentary observation 利用瞬时观测预测行人过街意图的长短期观测驱动预测网络

IF 5.5 2区计算机科学

Neurocomputing Pub Date : 2024-11-08 DOI: 10.1016/j.neucom.2024.128824

Hui Liu, Chunsheng Liu, Faliang Chang, Yansha Lu, Minhang Liu

{"title":"Long–Short Observation-driven Prediction Network for pedestrian crossing intention prediction with momentary observation","authors":"Hui Liu, Chunsheng Liu, Faliang Chang, Yansha Lu, Minhang Liu","doi":"10.1016/j.neucom.2024.128824","DOIUrl":"10.1016/j.neucom.2024.128824","url":null,"abstract":"<div><div>Pedestrian crossing intention prediction aims to predict whether the pedestrian will cross the road, which is crucial for the decision-making of intelligent vehicles and ensuring traffic safety. Existing methods just rely on long-term observation and rarely consider it challenging to obtain sufficiently long and precise observation in real-world scenarios. Focus on momentary observation, which only contains two frames of the preceding and current time, we propose a novel <em>Long–Short Observation-driven Prediction Network</em> (LSOP-Net). LSOP-Net comprises two critical components, the <em>Momentary Observation feature Extraction Module</em> (MOE-Module) and the <em>Multimodal Long–Short-term feature Fusion Module</em> (MLSFusion). Utilizing a hybrid training strategy and an external long-term feature pool, the MOE-Module is proposed to extract features with long-term patterns from momentary observations, which effectively mitigates feature deficiency arising from momentary observations. Based on a feature selection fusion mechanism, the MLSFusion is proposed to explicitly model the importance relationship between various modalities’ long–short-term features and the output, which adaptively fuses the long–short-term features from various modalities. Experimental results on the JAAD and PIE datasets demonstrate that our approach achieves superior performance in pedestrian crossing intention prediction with momentary observation.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"614 ","pages":"Article 128824"},"PeriodicalIF":5.5,"publicationDate":"2024-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142660579","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Improving semi-autoregressive machine translation with the guidance of syntactic dependency parsing structure 以句法依存解析结构为指导改进半自回归机器翻译

IF 5.5 2区计算机科学

Neurocomputing Pub Date : 2024-11-08 DOI: 10.1016/j.neucom.2024.128828

Xinran Chen, Sufeng Duan, Gongshen Liu

{"title":"Improving semi-autoregressive machine translation with the guidance of syntactic dependency parsing structure","authors":"Xinran Chen, Sufeng Duan, Gongshen Liu","doi":"10.1016/j.neucom.2024.128828","DOIUrl":"10.1016/j.neucom.2024.128828","url":null,"abstract":"<div><div>The advent of non-autoregressive machine translation (NAT) accelerates the decoding superior to autoregressive machine translation (AT) significantly, while bringing about a performance decrease. Semi-autoregressive neural machine translation (SAT), as a compromise, enjoys the merits of both autoregressive and non-autoregressive decoding. However, current SAT methods face the challenges of information-limited initialization and rigorous termination. This paper develops a layer-and-length-based syntactic labeling method and introduces a syntactic dependency parsing structure-guided two-stage semi-autoregressive translation (SDPSAT) structure, which addresses the above challenges with a syntax-based initialization and termination. Additionally, we also present a Mixed Training strategy to shrink exposure bias. Experiments on seven widely-used datasets reveal that our SDPSAT surpasses traditional SAT models with reduced word repetition and achieves competitive results with the AT baseline at a <span><math><mrow><mn>2</mn><mo>×</mo><mo>∼</mo><mn>3</mn><mo>×</mo></mrow></math></span> speedup.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"614 ","pages":"Article 128828"},"PeriodicalIF":5.5,"publicationDate":"2024-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142660670","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

HyV-Summ: Social media video summarization on custom dataset using hybrid techniques HyV-Summ：利用混合技术在定制数据集上进行社交媒体视频摘要

IF 5.5 2区计算机科学

Neurocomputing Pub Date : 2024-11-08 DOI: 10.1016/j.neucom.2024.128852

Jayanta Paul , Anuska Roy , Abhijit Mitra, Jaya Sil

{"title":"HyV-Summ: Social media video summarization on custom dataset using hybrid techniques","authors":"Jayanta Paul , Anuska Roy , Abhijit Mitra, Jaya Sil","doi":"10.1016/j.neucom.2024.128852","DOIUrl":"10.1016/j.neucom.2024.128852","url":null,"abstract":"<div><div>The proliferation of social networking platforms such as YouTube, Facebook, Instagram, and X has led to an exponential growth in multimedia content, with billions of videos uploaded every hour. Efficient management of such vast amount of data necessitates advanced summarization techniques in order to eliminate irrelevant and redundant information. A summarized video, containing the most distinct frames or key frames, provides a concise representation of the original content. Existing deep learning and non-deep learning techniques for video summarization have certain limitations. Deep learning methods are complex and resource-intensive, while non-deep learning algorithms often fail to extract informative features from vast social media videos. This paper addresses the issue by proposing a novel hybrid technique, named Hybrid Video Summarization (<strong>HyV-Summ</strong>), which integrates deep and non-deep learning techniques to leverage their respective strengths by focusing only on social media content. We developed a custom dataset, <strong>SocialSum</strong> to train our proposed model <strong>HyV-Summ</strong>, since existing benchmark datasets like TVSum and SumMe contain diverse types of content not specific to social media videos. We provide a comparative analysis of existing techniques and datasets with our proposed techniques and dataset. The results demonstrate that HyV-Summ outperforms existing techniques, such as Long Short Term Memory (LSTM)-based and Generative Adversarial Network (GAN)-based summarization by achieving higher F1-scores while applied on both the SocialSum dataset and available datasets.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"614 ","pages":"Article 128852"},"PeriodicalIF":5.5,"publicationDate":"2024-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142660757","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Deep feature response discriminative calibration 深度特征响应判别校准

IF 5.5 2区计算机科学

Neurocomputing Pub Date : 2024-11-08 DOI: 10.1016/j.neucom.2024.128848

Wenxiang Xu , Tian Qiu , Linyun Zhou , Zunlei Feng , Mingli Song , Huiqiong Wang

{"title":"Deep feature response discriminative calibration","authors":"Wenxiang Xu , Tian Qiu , Linyun Zhou , Zunlei Feng , Mingli Song , Huiqiong Wang","doi":"10.1016/j.neucom.2024.128848","DOIUrl":"10.1016/j.neucom.2024.128848","url":null,"abstract":"<div><div>Deep neural networks (DNNs) have numerous applications across various domains. Several optimization techniques, such as ResNet and SENet, have been proposed to improve model accuracy. These techniques improve the model performance by adjusting or calibrating feature responses according to a uniform standard. However, they lack the discriminative calibration for different features, thereby introducing limitations in the model output. Therefore, we propose a method that discriminatively calibrates feature responses. The preliminary experimental results indicate that the neural feature response follows a Gaussian distribution. Consequently, we compute confidence values by employing the Gaussian probability density function, and then integrate these values with the original response values. The objective of this integration is to improve the feature discriminability of the neural feature response. Based on the calibration values, we propose a plugin-based calibration module incorporated into a modified ResNet architecture, termed Response Calibration Networks (ResCNet). Extensive experiments on datasets like CIFAR-10, CIFAR-100, SVHN, and ImageNet demonstrate the effectiveness of the proposed approach.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"615 ","pages":"Article 128848"},"PeriodicalIF":5.5,"publicationDate":"2024-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142652693","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Multi-modal soft prompt-tuning for Chinese Clickbait Detection 针对中文点击诱饵检测的多模式软提示调整

IF 5.5 2区计算机科学

Neurocomputing Pub Date : 2024-11-08 DOI: 10.1016/j.neucom.2024.128829

Ye Wang , Yi Zhu , Yun Li , Liting Wei , Yunhao Yuan , Jipeng Qiang

{"title":"Multi-modal soft prompt-tuning for Chinese Clickbait Detection","authors":"Ye Wang , Yi Zhu , Yun Li , Liting Wei , Yunhao Yuan , Jipeng Qiang","doi":"10.1016/j.neucom.2024.128829","DOIUrl":"10.1016/j.neucom.2024.128829","url":null,"abstract":"<div><div>With the rapid growth of Chinese online services, clickbait has proliferated at an unprecedented rate, designed to manipulate users into clicking for increased traffic or advertising promotion. Such clickbait not only facilitates the spread of fake news and misinformation but also enables click-jacking attacks, redirecting users to deceptive websites that steal personal information. These harmful activities can result in significant losses and serious repercussions. The widespread presence of clickbait underscores both the importance and the challenges of developing effective detection methods. To date, the research paradigm of clickbait detection evolved from deep neural networks to fine-tuned Pre-trained Language Models (PLMs) and, more recently, into prompt-tuning models. However, these methods may suffer two main limitations: (1) they fail to utilize the multi-modal context information in news or posts and explore the higher-level feature representations to enhance the performance of clickbait detection; (2) they largely ignore the diverse range of Chinese expressive forms and neglect the complex semantics and syntactic structures of textual content to assist in learning a better news representation. To overcome these limitations, we proposed a Multi-modal Soft Prompt-tuning Method (MSP) for Chinese Clickbait Detection, which jointly models the textual and image information into a continuous prompt embedding as the input of PLMs. Specifically, firstly, the soft prompt-tuning model including Graph Attention Network and Contrastive Language-Image Pre-training are employed to learn the feature representations of texts and images in news or posts, respectively. Then the obtained text and image representations are re-input into the soft prompt-tuning model with automatic template generation. The extensive experiments on three Chinese clickbait detection datasets demonstrate that our MSP achieved state-of-the-art performance.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":"614 ","pages":"Article 128829"},"PeriodicalIF":5.5,"publicationDate":"2024-11-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142660662","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Quantifying neural network uncertainty under volatility clustering 量化波动聚类下的神经网络不确定性

IF 5.5 2区计算机科学

Neurocomputing Pub Date : 2024-11-07 DOI: 10.1016/j.neucom.2024.128816

Steven Y.K. Wong , Jennifer S.K. Chan , Lamiae Azizi

引用次数: 0