Neurocomputing最新文献_第2页

A review of AI edge devices and lightweight CNN and LLM deployment 人工智能边缘设备及轻量级 CNN 和 LLM 部署回顾

IF 5.5 2区计算机科学

Neurocomputing Pub Date : 2024-10-28 DOI: 10.1016/j.neucom.2024.128791

引用次数: 0

IoU-guided Siamese network with high-confidence template fusion for visual tracking 用于视觉跟踪的高置信度模板融合 IoU 引导连体网络

IF 5.5 2区计算机科学

Neurocomputing Pub Date : 2024-10-28 DOI: 10.1016/j.neucom.2024.128774

引用次数: 0

Auditing privacy budget of differentially private neural network models 审核不同隐私神经网络模型的隐私预算

IF 5.5 2区计算机科学

Neurocomputing Pub Date : 2024-10-28 DOI: 10.1016/j.neucom.2024.128756

{"title":"Auditing privacy budget of differentially private neural network models","authors":"","doi":"10.1016/j.neucom.2024.128756","DOIUrl":"10.1016/j.neucom.2024.128756","url":null,"abstract":"<div><div>In recent years, neural network models are used in various tasks. To eliminate privacy concern, differential privacy (DP) is introduced to the training phase of neural network models. However, introducing DP into neural network models is very subtle and error-prone, resulting in that some differentially private neural network models may not achieve privacy guarantee claimed. In this paper, we propose a method, which can audit privacy budget of differentially private neural network models. The proposed method is general and can be used to audit some other AI models. We elaborate on how to audit privacy budget of basic DP mechanisms and neural network models by the proposed method first. Then, we run experiments to verify our method. Experiment results indicate that the proposed method is better than the advanced method and the auditing precise is high when the privacy budget is small. In particular, when auditing privacy budget of ResNet-18 over CIFAR-10 protected by the differentially private mechanism with theoretical privacy budget 0.2, the accuracy of our method is about 17 times that of the state-of-the-art method. For the simpler dataset FMNIST, the accuracy of our method is about 32 times that of the state-of-the-art method when theoretical privacy budget is 0.2.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":null,"pages":null},"PeriodicalIF":5.5,"publicationDate":"2024-10-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142578802","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Subclass consistency regularization for learning with noisy labels based on contrastive learning 基于对比学习的噪声标签学习的子类一致性正则化

IF 5.5 2区计算机科学

Neurocomputing Pub Date : 2024-10-28 DOI: 10.1016/j.neucom.2024.128759

{"title":"Subclass consistency regularization for learning with noisy labels based on contrastive learning","authors":"","doi":"10.1016/j.neucom.2024.128759","DOIUrl":"10.1016/j.neucom.2024.128759","url":null,"abstract":"<div><div>A prominent effect of label noise on neural networks is the disruption of the consistency of predictions. While prior efforts primarily focused on predictions’ consistency at the individual instance level, they often fell short of fully harnessing the consistency across multiple instances. This paper introduces subclass consistency regularization (SCCR) to maximize the potential of this collective consistency of predictions. SCCR mitigates the impact of label noise on neural networks by imposing constraints on the consistency of predictions within each subclass. However, constructing high-quality subclasses poses a formidable challenge, which we formulate as a special clustering problem. To efficiently establish these subclasses, we incorporate a clustering-based contrastive learning framework. Additionally, we introduce the <span><math><mi>Q</mi></math></span>-enhancing algorithm to tailor the contrastive learning framework, ensuring alignment with subclass construction. We conducted comprehensive experiments using benchmark datasets and real datasets to evaluate the effectiveness of our proposed method under various scenarios with differing noise rates. The results unequivocally demonstrate the enhancement in classification accuracy, especially in challenging high-noise settings. Moreover, the refined contrastive learning framework significantly elevates the quality of subclasses even in the presence of noise. Furthermore, we delve into the compatibility of contrastive learning and learning with noisy labels, using the projection head as an illustrative example. This investigation sheds light on an aspect that has hitherto been overlooked in prior research efforts.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":null,"pages":null},"PeriodicalIF":5.5,"publicationDate":"2024-10-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142592861","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

ERAT-DLoRA: Parameter-efficient tuning with enhanced range adaptation in time and depth aware dynamic LoRA ERAT-DLoRA：在时间和深度感知动态 LoRA 中通过增强范围自适应进行参数高效调整

IF 5.5 2区计算机科学

Neurocomputing Pub Date : 2024-10-28 DOI: 10.1016/j.neucom.2024.128778

{"title":"ERAT-DLoRA: Parameter-efficient tuning with enhanced range adaptation in time and depth aware dynamic LoRA","authors":"","doi":"10.1016/j.neucom.2024.128778","DOIUrl":"10.1016/j.neucom.2024.128778","url":null,"abstract":"<div><div>Despite their potential, the industrial deployment of large language models (LLMs) is constrained by traditional fine-tuning procedures that are both resource-intensive and time-consuming. Low-Rank Adaptation (LoRA) has emerged as a pioneering methodology for addressing these challenges. By integrating low-rank decomposition matrices into network weights to reduce trainable parameters, LoRA effectively accelerates the adaptation process. While research on LoRA primarily focuses on adjusting low-rank matrices, DyLoRA optimizes the rank-setting mechanism to avoid extensive effort in rank size training and searching. However, DyLoRA rank configuration mechanism has its own limitation. First, DyLoRA sets the same rank size for all the low-rank adaptation layers at each time step. Given that layers with different depth contain distinct information, they should have varying rank values to accurately capture their unique characteristics. Second, the truncated phase selected for ordering representation based on nested dropout regulation is only half dynamic, continuously dropping tail units, thereby limiting its ability to access information. In this work, we propose a novel technique, enhanced range adaptation in time and depth aware dynamic LoRA (ERAT-DLoRA) to address these problems. The ERAT-DLoRA method introduces a dynamic range to the truncated phase that makes the truncated phase fully dynamic. Additionally, we design a time and layer-aware dynamic rank to ensure appropriate adjustments at different time steps and layer levels. We evaluate our solution on natural languages understanding and language generation tasks. Extensive evaluation results demonstrate the effectiveness of the proposed method.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":null,"pages":null},"PeriodicalIF":5.5,"publicationDate":"2024-10-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142586926","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Observer-based adaptive neural network event-triggered quantized control for active suspensions with actuator saturation 基于观测器的自适应神经网络事件触发量化控制，用于具有致动器饱和度的主动悬挂系统

IF 5.5 2区计算机科学

Neurocomputing Pub Date : 2024-10-28 DOI: 10.1016/j.neucom.2024.128770

{"title":"Observer-based adaptive neural network event-triggered quantized control for active suspensions with actuator saturation","authors":"","doi":"10.1016/j.neucom.2024.128770","DOIUrl":"10.1016/j.neucom.2024.128770","url":null,"abstract":"<div><div>This paper proposes an adaptive neural network event-triggered and quantized output feedback control scheme for quarter vehicle active suspensions with actuator saturation. The scheme uses neural networks to approximate the unknown parts of the active suspension. When the system states of the suspension are not entirely available, a state observer is designed to estimate the unknown states. The measurable system states, partially estimated observer states, neural network weights, and a filtered virtual control are sequentially event-triggered, quantified, and transmitted to the controller via in-vehicle networks. The problem of non-differentiable virtual control is solved using dynamic surface control technology in the backstepping quantized control design. Integrating a Gaussian error function and a first-order auxiliary subsystem compensates for the nonlinearity caused by asymmetric saturation. Theoretical analysis proves that all error signals of the closed-loop active suspension system are semi-globally uniformly ultimately bounded, and the Zeno phenomenon can be ruled out. Simulation results validate the effectiveness of the proposed control method.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":null,"pages":null},"PeriodicalIF":5.5,"publicationDate":"2024-10-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142573193","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A three-stage model for camouflaged object detection 伪装物体检测的三阶段模型

IF 5.5 2区计算机科学

Neurocomputing Pub Date : 2024-10-28 DOI: 10.1016/j.neucom.2024.128784

{"title":"A three-stage model for camouflaged object detection","authors":"","doi":"10.1016/j.neucom.2024.128784","DOIUrl":"10.1016/j.neucom.2024.128784","url":null,"abstract":"<div><div>Camouflaged objects are typically assimilated into their backgrounds and exhibit fuzzy boundaries. The complex environmental conditions and the high intrinsic similarity between camouflaged targets and their surroundings pose significant challenges in accurately locating and segmenting these objects in their entirety. While existing methods have demonstrated remarkable performance in various real-world scenarios, they still face limitations when confronted with difficult cases, such as small targets, thin structures, and indistinct boundaries. Drawing inspiration from human visual perception when observing images containing camouflaged objects, we propose a three-stage model that enables coarse-to-fine segmentation in a single iteration. Specifically, our model employs three decoders to sequentially process subsampled features, cropped features, and high-resolution original features. This proposed approach not only reduces computational overhead but also mitigates interference caused by background noise. Furthermore, considering the significance of multi-scale information, we have designed a multi-scale feature enhancement module that enlarges the receptive field while preserving detailed structural cues. Additionally, a boundary enhancement module has been developed to enhance performance by leveraging boundary information. Subsequently, a mask-guided fusion module is proposed to generate fine-grained results by integrating coarse prediction maps with high-resolution feature maps. Our network shows superior performance without introducing unnecessary complexities. Upon acceptance of the paper, the source code will be made publicly available at <span><span>https://github.com/clelouch/TSNet</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":null,"pages":null},"PeriodicalIF":5.5,"publicationDate":"2024-10-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142573294","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

MHEC: One-shot relational learning of knowledge graphs completion based on multi-hop information enhancement MHEC：基于多跳信息增强的知识图谱完成的一次性关系学习

IF 5.5 2区计算机科学

Neurocomputing Pub Date : 2024-10-28 DOI: 10.1016/j.neucom.2024.128760

{"title":"MHEC: One-shot relational learning of knowledge graphs completion based on multi-hop information enhancement","authors":"","doi":"10.1016/j.neucom.2024.128760","DOIUrl":"10.1016/j.neucom.2024.128760","url":null,"abstract":"<div><div>With the wide application of knowledge graphs, knowledge graph completion has garnered increasing attention in recent years. However, we find that the long tail relation is more common in the KG. These relations typically do not have a large number of triples for training and are referred to as few-shot relations. The knowledge graph completion in the few-shot scenario is a major challenge currently. The current mainstream knowledge graph completion algorithms have the following drawbacks. The metric-based methods lack interpretability of results, while the algorithms based on path interaction are not suitable for few-shot scenarios and the availability of the model is limited in sparse knowledge graphs. In this paper, we propose a one-shot relational learning of knowledge graphs completion based on multi-hop information enhancement(MHEC). Firstly, MHEC extracts entity concepts from multi-hop paths to obtain task related entity concepts and filters out noisy neighbor attributes. Then, MHEC combines multi-hop path information between head and tail to represent entity pairs. Compared to previous completion methods that only consider structural features of entities, MHEC considers the reasoning logic between entity pairs, which not only includes structural features but also possesses rich semantic features. Next, MHEC introduces a reasoning process in the completion task to address the issues of lack of interpretability in the one-shot scenario. In addition, to improve completion and reasoning quality in sparse knowledge graphs, MHEC utilizes contrastive learning to enhance pre-training vector representations of entities and relations and proposes a matching processor that leverages the semantic information of pre-training vectors to assist the reasoning model in expanding the multi-hop paths. Experiments demonstrate that MHEC outperforms the state-of-the-art completion techniques on real-world datasets NELL-One and FB15k237-One.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":null,"pages":null},"PeriodicalIF":5.5,"publicationDate":"2024-10-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142573297","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

CRISP: A cross-modal integration framework based on the surprisingly popular algorithm for multimodal named entity recognition CRISP：基于多模态命名实体识别惊人流行算法的跨模态整合框架

IF 5.5 2区计算机科学

Neurocomputing Pub Date : 2024-10-28 DOI: 10.1016/j.neucom.2024.128792

{"title":"CRISP: A cross-modal integration framework based on the surprisingly popular algorithm for multimodal named entity recognition","authors":"","doi":"10.1016/j.neucom.2024.128792","DOIUrl":"10.1016/j.neucom.2024.128792","url":null,"abstract":"<div><div>The multimodal named entity recognition task on social media involves recognizing named entities with textual and visual information, which is of great significance for information processing. Nevertheless, many existing models still face the following challenges. First, in the process of cross-modal interaction, the attention mechanism sometimes focuses on trivial parts in the images that are not relevant to entities, which not only neglects valuable information but also inevitably introduces visual noise. Second, the gate mechanism is widely used for filtering out visual information to reduce the influence of noise on text understanding. However, the gate mechanism neglects capturing fine-grained semantic relevance between modalities, which easily affects the filtration process. To address these issues, we propose a cross-modal integration framework based on the surprisingly popular algorithm, aiming at enhancing the integration of effective visual guidance and reducing the interference of irrelevant visual noise. Specifically, we design a dual-branch interaction module that includes the attention mechanism and the surprisingly popular algorithm, allowing the model to focus on valuable but overlooked parts in the images. Furthermore, we compute the matching degree between modalities at the multi-granularity level, using the Choquet integral to establish a more reasonable basis for filtering out visual noise. We have conducted extensive experiments on public datasets, and the experimental results demonstrate the advantages of our model.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":null,"pages":null},"PeriodicalIF":5.5,"publicationDate":"2024-10-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142578800","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Robust source-free domain adaptation with anti-adversarial samples training 通过反对抗样本训练实现稳健的无源领域适应性

IF 5.5 2区计算机科学

Neurocomputing Pub Date : 2024-10-28 DOI: 10.1016/j.neucom.2024.128777

{"title":"Robust source-free domain adaptation with anti-adversarial samples training","authors":"","doi":"10.1016/j.neucom.2024.128777","DOIUrl":"10.1016/j.neucom.2024.128777","url":null,"abstract":"<div><div>Unsupervised source-free domain adaptation methods aim to transfer knowledge acquired from labeled source domain to an unlabeled target domain, where the source data are not accessible during target domain adaptation and it is prohibited to minimize domain gap by pairwise calculation of the samples from the source and target domains. Previous approaches assign pseudo label to target data using pre-trained source model to progressively train the target model in a self-learning manner. However, incorrect pseudo label may adversely affect prediction in the target domain. Furthermore, they overlook the generalization ability of the source model, which primarily affects the initial prediction of the target model. Therefore, we propose an effective framework based on adversarial training to train the target model for source-free domain adaptation. Specifically, adversarial training is an effective technique to enhance the robustness of deep neural networks. By generating anti-adversarial examples and adversarial examples, the pseudo label of target data can be corrected further by adversarial training and a more optimal performance in both accuracy and robustness is achieved. Moreover, owing to the inherent domain distribution difference between source and target domains, mislabeled target samples exist inevitably. So a target sample filtering scheme is proposed to refine pseudo label to further improve the prediction capability on the target domain. Experiments conducted on benchmark tasks demonstrate that the proposed method outperforms existing approaches.</div></div>","PeriodicalId":19268,"journal":{"name":"Neurocomputing","volume":null,"pages":null},"PeriodicalIF":5.5,"publicationDate":"2024-10-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142586924","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0