Neural Networks最新文献_第5页

Pyramid contrastive learning for clustering

IF 6 1区计算机科学

Neural Networks Pub Date : 2025-02-04 DOI: 10.1016/j.neunet.2025.107217

Zi-Feng Zhou , Dong Huang , Chang-Dong Wang

{"title":"Pyramid contrastive learning for clustering","authors":"Zi-Feng Zhou , Dong Huang , Chang-Dong Wang","doi":"10.1016/j.neunet.2025.107217","DOIUrl":"10.1016/j.neunet.2025.107217","url":null,"abstract":"<div><div>With its ability of joint representation learning and clustering via deep neural networks, the deep clustering have gained significant attention in recent years. Despite the considerable progress, most of the previous deep clustering methods still suffer from three critical limitations. First, they tend to associate some distribution-based clustering loss to the neural network, which often overlook the sample-wise contrastiveness for discriminative representation learning. Second, they generally utilize the features learned at a single layer for the clustering process, which, surprisingly, cannot go beyond a single layer to explore multiple layers for joint multi-layer (multi-stage) learning. Third, they typically use the convolutional neural network (CNN) for clustering images, which focus on local information yet cannot well capture the global dependencies. To tackle these issues, this paper presents a new deep clustering method called pyramid contrastive learning for clustering (PCLC), which is able to incorporate a pyramidal contrastive architecture to jointly enforce contrastive learning and clustering at multiple network layers (or stages). Particularly, for an input image, two types of augmentations are first performed to generate two paralleled augmented views. To bridge the gap between the CNN (for capturing local information) and the Transformer (for reflecting global dependencies), a mixed CNN-Transformer based encoder is utilized as the backbone, whose CNN-Transformer blocks are further divided into four stages, thus giving rise to a pyramid of multi-stage feature representations. Thereafter, multiple stages of twin contrastive learning are simultaneously conducted at both the instance-level and the cluster-level, through the optimization of which the final clustering can be achieved. Extensive experiments on multiple challenging image datasets demonstrate the superior clustering performance of PCLC over the state-of-the-art. The source code is available at <span><span>https://github.com/Zachary-Chow/PCLC</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"185 ","pages":"Article 107217"},"PeriodicalIF":6.0,"publicationDate":"2025-02-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143348201","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

GTIGNet: Global Topology Interaction Graphormer Network for 3D hand pose estimation

IF 6 1区计算机科学

Neural Networks Pub Date : 2025-02-04 DOI: 10.1016/j.neunet.2025.107221

Yanjun Liu , Wanshu Fan , Cong Wang , Shixi Wen , Xin Yang , Qiang Zhang , Xiaopeng Wei , Dongsheng Zhou

{"title":"GTIGNet: Global Topology Interaction Graphormer Network for 3D hand pose estimation","authors":"Yanjun Liu , Wanshu Fan , Cong Wang , Shixi Wen , Xin Yang , Qiang Zhang , Xiaopeng Wei , Dongsheng Zhou","doi":"10.1016/j.neunet.2025.107221","DOIUrl":"10.1016/j.neunet.2025.107221","url":null,"abstract":"<div><div>Estimating 3D hand poses from monocular RGB images presents a series of challenges, including complex hand structures, self-occlusions, and depth ambiguities. Existing methods often fall short of capturing the long-distance dependencies of skeletal and non-skeletal connections for hand joints. To address these limitations, we introduce the Global Topology Interaction Graphormer Network (GTIGNet), a novel deep learning architecture designed to improve 3D hand pose estimation. Our model incorporates a Context-Aware Attention Block (CAAB) within the 2D pose estimator to enhance the extraction of multi-scale features, yielding more accurate 2D joint heatmaps to support the task that followed. Additionally, we introduce a High-Order Graphormer that explicitly and implicitly models the topological structure of hand joints, thereby enhancing feature interaction. Ablation studies confirm the effectiveness of our approach, and experimental results on four challenging datasets, Rendered Hand Dataset (RHD), Stereo Hand Pose Benchmark (STB), First-Person Hand Action Benchmark (FPHA), and FreiHAND Dataset, indicate that GTIGNet achieves state-of-the-art performance in 3D hand pose estimation. Notably, our model achieves an impressive Mean Per Joint Position Error (MPJPE) of 9.98 mm on RHD, 6.12 mm on STB, 11.15 mm on FPHA and 10.97 mm on FreiHAND.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"185 ","pages":"Article 107221"},"PeriodicalIF":6.0,"publicationDate":"2025-02-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143349465","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Semi-supervised learning for multi-view and non-graph data using Graph Convolutional Networks

IF 6 1区计算机科学

Neural Networks Pub Date : 2025-02-03 DOI: 10.1016/j.neunet.2025.107218

F. Dornaika , J. Bi , J. Charafeddine , H. Xiao

{"title":"Semi-supervised learning for multi-view and non-graph data using Graph Convolutional Networks","authors":"F. Dornaika , J. Bi , J. Charafeddine , H. Xiao","doi":"10.1016/j.neunet.2025.107218","DOIUrl":"10.1016/j.neunet.2025.107218","url":null,"abstract":"<div><div>Semi-supervised learning with a graph-based approach has become increasingly popular in machine learning, particularly when dealing with situations where labeling data is a costly process. Graph Convolution Networks (GCNs) have been widely employed in semi-supervised learning, primarily on graph-structured data like citations and social networks. However, there exists a significant gap in applying these methods to non-graph multi-view data, such as collections of images. To bridge this gap, we introduce a novel deep semi-supervised multi-view classification model tailored specifically for non-graph data. This model independently reconstructs individual graphs using a powerful semi-supervised approach and subsequently merges them adaptively into a unified consensus graph. The consensus graph feeds into a unified GCN framework incorporating a label smoothing constraint.</div><div>To assess the efficacy of the proposed model, experiments were conducted across seven multi-view image datasets. Results demonstrate that this model excels in both the graph generation and semi-supervised classification phases, consistently outperforming classical GCNs and other existing semi-supervised multi-view classification approaches. <span><span><sup>1</sup></span></span></div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"185 ","pages":"Article 107218"},"PeriodicalIF":6.0,"publicationDate":"2025-02-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143349461","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Unifying and revisiting Sharpness-Aware Minimization with noise-injected micro-batch scheduler for efficiency improvement

IF 6 1区计算机科学

Neural Networks Pub Date : 2025-02-03 DOI: 10.1016/j.neunet.2025.107205

Zheng Wei, Xingjun Zhang, Zhendong Tan

{"title":"Unifying and revisiting Sharpness-Aware Minimization with noise-injected micro-batch scheduler for efficiency improvement","authors":"Zheng Wei, Xingjun Zhang, Zhendong Tan","doi":"10.1016/j.neunet.2025.107205","DOIUrl":"10.1016/j.neunet.2025.107205","url":null,"abstract":"<div><div>Sharpness-aware minimization (SAM) has been proposed to improve generalization by encouraging the model to converge to a flatter region. However, SAM’s two sequential gradient computations lead to 2<span><math><mo>×</mo></math></span> computation overhead compared to the base optimizer (e.g., SGD). Recent works improve SAM’s efficiency either by switching between SAM and base optimizer or by reducing data samples. In this paper, we first propose the micro-batch scheduler to unify the above two ideas and summarize that the commonality of them is adopting a smaller micro-batch to approximate the perturbation. However, its role is not fully explored. Thus, we revisit the effect of micro-batch approximated perturbation on accuracy and efficiency and empirically observe that a too-small micro-batch causes accuracy degradation as it leads to a sharper loss landscape. To alleviate it, we inject random noise into the micro-batch approximated gradient in SAM’s first ascent step, which implicitly leverages random perturbation before SAM’s second descent step. The visualization results confirm that it encourages the model to converge to a flatter region. Extensive experiments with various models (e.g., ResNet-18/50, WideResNet-28-10, PyramidNet-110, and ViT-B/16, etc.) evaluated on CIFAR-10 and ImageNet-1K show that the proposed method achieves competitive accuracy with higher efficiency when compared to several efficient SAM variants (e.g., ESAM, LooKSAM-5, AE-SAM, K-SAM, etc.).</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"185 ","pages":"Article 107205"},"PeriodicalIF":6.0,"publicationDate":"2025-02-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143349460","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Break Adhesion: Triple adaptive-parsing for weakly supervised instance segmentation

IF 6 1区计算机科学

Neural Networks Pub Date : 2025-02-01 DOI: 10.1016/j.neunet.2025.107215

Jingting Xu , Rui Cao , Peng Luo , Dejun Mu

{"title":"Break Adhesion: Triple adaptive-parsing for weakly supervised instance segmentation","authors":"Jingting Xu , Rui Cao , Peng Luo , Dejun Mu","doi":"10.1016/j.neunet.2025.107215","DOIUrl":"10.1016/j.neunet.2025.107215","url":null,"abstract":"<div><div>Weakly supervised instance segmentation (WSIS) aims to identify individual instances from weakly supervised semantic segmentation precisely. Existing WSIS techniques primarily employ a unified, fixed threshold to identify all peaks in semantic maps. It may lead to potential missed or false detections due to the same category but with diverse visual characteristics. Moreover, previous methods apply a fixed augmentation strategy to broadly propagate peak cues to contributing regions, resulting in instance adhesion. To eliminate these manually fixed parsing patterns, we propose a triple adaptive-parsing network. Specifically, an adaptive Peak Perception Module (PPM) employs the average degree of feature as a learning base to infer the optimal threshold. Simultaneously, we propose the Shrinkage Loss function (SL) to minimize outlier responses that deviate from the mean. Finally, by eliminating uncertain adhesion, our method effectively obtains Reliable Inter-instance Relationships (RIR), enhancing the representation of instances. Extensive experiments on the Pascal VOC and COCO datasets show that the proposed method improves the accuracy by 2.1% and 4.3%, achieving the latest performance standard and significantly optimizing the instance segmentation task. The code is available at <span><span>https://github.com/Elaineok/TAP</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"186 ","pages":"Article 107215"},"PeriodicalIF":6.0,"publicationDate":"2025-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143395985","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

SDR-Former: A Siamese Dual-Resolution Transformer for liver lesion classification using 3D multi-phase imaging

IF 6 1区计算机科学

Neural Networks Pub Date : 2025-02-01 DOI: 10.1016/j.neunet.2025.107228

Meng Lou , Hanning Ying , Xiaoqing Liu , Hong-Yu Zhou , Yuqin Zhang , Yizhou Yu

{"title":"SDR-Former: A Siamese Dual-Resolution Transformer for liver lesion classification using 3D multi-phase imaging","authors":"Meng Lou , Hanning Ying , Xiaoqing Liu , Hong-Yu Zhou , Yuqin Zhang , Yizhou Yu","doi":"10.1016/j.neunet.2025.107228","DOIUrl":"10.1016/j.neunet.2025.107228","url":null,"abstract":"<div><div>Automated classification of liver lesions in multi-phase CT and MR scans is of clinical significance but challenging. This study proposes a novel Siamese Dual-Resolution Transformer (SDR-Former) framework, specifically designed for liver lesion classification in 3D multi-phase CT and MR imaging with varying phase counts. The proposed SDR-Former utilizes a streamlined Siamese Neural Network (SNN) to process multi-phase imaging inputs, possessing robust feature representations while maintaining computational efficiency. The weight-sharing feature of the SNN is further enriched by a hybrid Dual-Resolution Transformer (DR-Former), comprising a 3D Convolutional Neural Network (CNN) and a tailored 3D Transformer for processing high- and low-resolution images, respectively. This hybrid sub-architecture excels in capturing detailed local features and understanding global contextual information, thereby, boosting the SNN’s feature extraction capabilities. Additionally, a novel Adaptive Phase Selection Module (APSM) is introduced, promoting phase-specific intercommunication and dynamically adjusting each phase’s influence on the diagnostic outcome. The proposed SDR-Former framework has been validated through comprehensive experiments on two clinically collected datasets: a 3-phase CT dataset and an 8-phase MR dataset. The experimental results affirm the efficacy of the proposed framework. To support the scientific community, we are releasing our extensive multi-phase MR dataset for liver lesion analysis to the public. This pioneering dataset, being the first publicly available multi-phase MR dataset in this field, also underpins the MICCAI LLD-MMRI Challenge. The dataset is publicly available at: <span><span>https://github.com/LMMMEng/LLD-MMRI-Dataset</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"185 ","pages":"Article 107228"},"PeriodicalIF":6.0,"publicationDate":"2025-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143140553","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

CURRENT EVENTS

IF 6 1区计算机科学

Neural Networks Pub Date : 2025-02-01 DOI: 10.1016/S0893-6080(24)00940-7

引用次数: 0

INN/ENNS/JNNS - Membership Applic. Form

IF 6 1区计算机科学

Neural Networks Pub Date : 2025-02-01 DOI: 10.1016/S0893-6080(24)00941-9

引用次数: 0

Multi-knowledge informed deep learning model for multi-point prediction of Alzheimer’s disease progression

IF 6 1区计算机科学

Neural Networks Pub Date : 2025-02-01 DOI: 10.1016/j.neunet.2025.107203

Kai Wu , Hong Wang , Feiyan Feng , Tianyu Liu , Yanshen Sun

{"title":"Multi-knowledge informed deep learning model for multi-point prediction of Alzheimer’s disease progression","authors":"Kai Wu , Hong Wang , Feiyan Feng , Tianyu Liu , Yanshen Sun","doi":"10.1016/j.neunet.2025.107203","DOIUrl":"10.1016/j.neunet.2025.107203","url":null,"abstract":"<div><div>The diagnosis of Alzheimer’s disease (AD) based on visual features-informed by clinical knowledge has achieved excellent results. Our study endeavors to present an innovative and detailed deep learning framework designed to accurately predict the progression of Alzheimer’s disease. We propose <strong>Mul-KMPP</strong>, a <strong>Mul</strong>ti-<strong>K</strong>nowledge Informed Deep Learning Model for <strong>M</strong>ulti-<strong>P</strong>oint <strong>P</strong>rediction of AD progression, intended to facilitate precise assessments of AD progression in older adults. Firstly, we designed a dual-path methodology to capture global and local brain characteristics for visual feature extraction (utilizing MRIs). Then, we developed a diagnostic module before the prediction module, leveraging AAL (Anatomical Automatic Labeling) knowledge. Following this, predictions are informed by clinical insights. For this purpose, we devised a new composite loss function, including diagnosis loss, prediction loss, and consistency loss of the two modules. To validate our model, we compiled a dataset comprising 819 samples and the results demonstrate that our Mul-KMPP model achieved an accuracy of 86.8%, sensitivity of 86.1%, specificity of 92.1%, and area under the curve (AUC) of 95.9%, significantly outperforming several competing diagnostic methods at every time point. The source code for our model is available at <span><span>https://github.com/Camelus-to/Mul-KMPP</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"185 ","pages":"Article 107203"},"PeriodicalIF":6.0,"publicationDate":"2025-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143349463","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

What is the impact of discrete memristor on the performance of neural network: A research on discrete memristor-based BP neural network

IF 6 1区计算机科学

Neural Networks Pub Date : 2025-02-01 DOI: 10.1016/j.neunet.2025.107213

Yuexi Peng , Maolin Li , Zhijun Li , Minglin Ma , Mengjiao Wang , Shaobo He

{"title":"What is the impact of discrete memristor on the performance of neural network: A research on discrete memristor-based BP neural network","authors":"Yuexi Peng , Maolin Li , Zhijun Li , Minglin Ma , Mengjiao Wang , Shaobo He","doi":"10.1016/j.neunet.2025.107213","DOIUrl":"10.1016/j.neunet.2025.107213","url":null,"abstract":"<div><div>Artificial neural networks are receiving increasing attention from researchers. However, with the advent of big data era, artificial neural networks are limited by the Von Neumann architecture, making it difficult to make new breakthroughs in hardware implementation. Discrete-time memristor, emerging as a research focus in recent years, are anticipated to address this challenge effectively. To enrich the theoretical research of memristors in artificial neural networks, this paper studies BP neural networks based on various discrete memristors. Firstly, the concept of discrete memristor and several classical discrete memristor models are introduced. Based on these models, the discrete memristor-based BP neural networks are designed. Finally, these networks are utilized for achieving handwritten digit classification and speech feature classification, respectively. The results show that linear discrete memristors perform better than nonlinear discrete memristors, and a simple linear discrete memristor-based BP neural network has the best performance, reaching 97.40% (handwritten digit classification) and 93.78% (speech feature classification), respectively. In addition, some fundamental issues are also discussed, such as the effects of linear, nonlinear memristors, and initial charges on the performance of neural networks.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"185 ","pages":"Article 107213"},"PeriodicalIF":6.0,"publicationDate":"2025-02-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143140551","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0