Neural Networks最新文献

{"title":"Data-free knowledge distillation via text-noise fusion and dynamic adversarial temperature","authors":"Deheng Zeng ,&nbsp;Zhengyang Wu ,&nbsp;Yunwen Chen ,&nbsp;Zhenhua Huang","doi":"10.1016/j.neunet.2025.108061","DOIUrl":"10.1016/j.neunet.2025.108061","url":null,"abstract":"<div><div>Data-Free Knowledge Distillation (DFKD) have achieved significant breakthroughs, enabling the effective transfer of knowledge from teacher neural networks to student neural networks without reliance on original data. However, a significant challenge faced by existing methods that attempt to generate samples from random noise is that the noise lacks meaningful information, such as class-specific semantic information. Consequently, the absence of meaningful information makes it difficult for the generator to map this noise to the ground-truth data distribution, resulting in the generation of low-quality training samples. In addition, existing methods typically employ a fixed temperature for adversarial training of the generator, which limits the diversity in the difficulty of the synthesized data. In this paper, we propose Text-Noise Fusion and Dynamic Adversarial Temperature method (TNFDAT), a novel method that combines random noise with meaningful class-specific text embeddings (CSTE) as input and implements dynamic adjustment of the adversarial training temperature for the generator. In addition, we introduce an adaptive sample weighting strategy to enhance the effectiveness of knowledge distillation. CSTE is developed based on a pre-trained language model, and its significance lies in its ability to capture meaningful inter-class information, thereby enabling the generation of high-quality samples. Simultaneously, the dynamic adversarial temperature module effectively alleviates the issue of insufficient diversity in synthesized samples by precisely modulating the generator’s temperature during adversarial training, playing a key role in enhancing sample diversity. Through continuous and dynamic temperature adjustment of the generator in the adversarial training, thereby significantly improving the overall diversity of the synthesized samples. At the knowledge distillation stage, We determine the distillation weights of the synthesized samples based on the information entropy of the output from both teacher and student networks. By differentiating the contributions of different synthesized samples during the distillation process, we effectively enhance the generalization ability of the knowledge distillation framework and improve the robustness of the student network. Experiments demonstrate that our method outperforms the state-of-the-art methods across various benchmarks and pairs of teachers and students.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"193 ","pages":"Article 108061"},"PeriodicalIF":6.3,"publicationDate":"2025-09-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145019292","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Mitigating low-frequency bias: Feature recalibration and frequency attention regularization for adversarial robustness","authors":"Kejia Zhang ,&nbsp;Juanjuan Weng ,&nbsp;Yuanzheng Cai ,&nbsp;Shaozi Li ,&nbsp;Zhiming Luo","doi":"10.1016/j.neunet.2025.108070","DOIUrl":"10.1016/j.neunet.2025.108070","url":null,"abstract":"<div><div>Ensuring the robustness of deep neural networks against adversarial attacks remains a fundamental challenge in computer vision. While adversarial training (AT) has emerged as a promising defense strategy, our analysis reveals a critical limitation: AT-trained models exhibit a bias toward low-frequency features while neglecting high-frequency components. This bias is particularly concerning as each frequency component carries distinct and crucial information: low-frequency features encode fundamental structural patterns, while high-frequency features capture intricate details and textures. To address this limitation, we propose High-Frequency Feature Disentanglement and Recalibration (HFDR), a novel module that strategically separates and recalibrates frequency-specific features to capture latent semantic cues. We further introduce frequency attention regularization to harmonize feature extraction across the frequency spectrum and mitigate the inherent low-frequency bias of AT. Extensive experiments on CIFAR-10, CIFAR-100, and ImageNet-1K demonstrate that HFDR consistently enhances adversarial robustness. It achieves a 2.89 % gain on CIFAR-100 with WRN34-10, and improves robustness by 3.09 % on ImageNet-1K, with a 4.89 % gain on ViT-B against AutoAttack. These results highlight the method’s adaptability to both convolutional and transformer-based architectures. Code is available at <span><span>https://github.com/KejiaZhang-Robust/HFDR</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"193 ","pages":"Article 108070"},"PeriodicalIF":6.3,"publicationDate":"2025-09-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145027022","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Auction-guided model diffusion for communication-efficient federated learning on non-IID data","authors":"Seyoung Ahn ,&nbsp;Soohyeong Kim ,&nbsp;Yongseok Kwon ,&nbsp;Jiseung Youn ,&nbsp;Joohan Park ,&nbsp;Sunghyun Cho","doi":"10.1016/j.neunet.2025.108066","DOIUrl":"10.1016/j.neunet.2025.108066","url":null,"abstract":"<div><div>In 6G mobile communication systems, various AI-based network functions and applications have been standardized. Federated learning (FL) is adopted as the core learning architecture for 6G systems to avoid privacy leakage from mobile user data. However, in FL, users with non-independent and identically distributed (non-IID) datasets can deteriorate the performance of the global model because the convergence direction of the gradient for each dataset is different, thereby inducing a weight divergence problem. To address this problem, we propose a novel diffusion strategy for machine learning (ML) models (FedDif) to maximize the performance of the global model with non-IID data. FedDif enables the local model to learn different distributions before parameter aggregation by passing the local models to users via device-to-device communication. Furthermore, we theoretically demonstrate that FedDif can circumvent the weight-divergence problem. Based on this theory, we propose a communication-efficient diffusion strategy for ML models that can determine the trade-off between learning performance and communication cost using auction theory. The experimental results show that FedDif improves the top-1 test accuracy by up to 20.07 %p and reduces communication costs by up to 45.27 % compared to FedAvg.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"193 ","pages":"Article 108066"},"PeriodicalIF":6.3,"publicationDate":"2025-09-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145019891","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"TATrack: Target-oriented adaptive vision transformer for UAV tracking","authors":"Wenkang Zhang ,&nbsp;Tianyang Xu ,&nbsp;Fei Xie ,&nbsp;Jinhui Wu ,&nbsp;Wankou Yang","doi":"10.1016/j.neunet.2025.108067","DOIUrl":"10.1016/j.neunet.2025.108067","url":null,"abstract":"<div><div>Unmanned Aerial Vehicle (UAV) tracking requires accurate target localization from aerial top-down perspectives while operating under the computational constraints of aerial platforms. Current mainstream UAV trackers, constrained by the limited resources, predominantly employ lightweight Convolutional Neural Network (CNN) extractor, coupled with an appearance-based fusion mechanism. The absence of comprehensive target perception significantly constrains the balance between tracking accuracy and computational efficiency. To address this, we propose a target-oriented adaptive vision transformer for UAV tracking, named TATrack. TATrack utilizes a novel efficient transformer model, TA-ViT, to perform joint feature modeling and interaction under the orientation of the target. Specifically, TA-ViT employs an adaptive scoring suspension mechanism, wherein redundant network layers are bypassed when all token scores meet the suspension criteria, thereby enhancing inference speed. Moreover, positional information is utilized as a spatial-temporal prompt to enhance appearance-matching quality over time. By introducing location priors, we strengthen the visual perception of the target, which improves the target orientation and temporal continuity of the predicted position. Extensive experiments conducted across five UAV tracking benchmarks demonstrate that our method achieves an optimal balance between computational efficiency and tracking accuracy. The code will be available publicly.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"193 ","pages":"Article 108067"},"PeriodicalIF":6.3,"publicationDate":"2025-09-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145019893","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Region-guided attack on the segment anything model","authors":"Xiaoliang Liu ,&nbsp;Furao Shen ,&nbsp;Jian Zhao","doi":"10.1016/j.neunet.2025.108058","DOIUrl":"10.1016/j.neunet.2025.108058","url":null,"abstract":"<div><div>The Segment Anything Model (SAM) is a cornerstone of image segmentation, demonstrating exceptional performance across various applications, particularly in autonomous driving and medical imaging, where precise segmentation is crucial. However, SAM is vulnerable to adversarial attacks that can significantly impair its functionality through minor input perturbations. Traditional techniques, such as FGSM and PGD, are often ineffective in segmentation tasks due to their reliance on global perturbations that overlook spatial nuances. Recent methods like Attack-SAM-K and UAD have begun to address these challenges, but they frequently depend on external cues and do not fully leverage the structural interdependencies within segmentation processes. This limitation underscores the need for a novel adversarial strategy that exploits the unique characteristics of segmentation tasks. In response, we introduce the Region-Guided Attack (RGA), designed specifically for SAM. RGA utilizes a Region-Guided Map (RGM) to manipulate segmented regions, enabling targeted perturbations that fragment large segments and expand smaller ones, resulting in erroneous outputs from SAM. Our experiments demonstrate that RGA achieves high success rates in both white-box and black-box scenarios, emphasizing the need for robust defenses against such sophisticated attacks. Our codes are available at <span><span>https://github.com/AbeLiuXL/RGA</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"193 ","pages":"Article 108058"},"PeriodicalIF":6.3,"publicationDate":"2025-09-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145010007","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Multimodal self-supervised retinal vessel segmentation","authors":"Pengshuai Yin ,&nbsp;Jingqi Zhang ,&nbsp;Huichou Huang ,&nbsp;Ruirui Liu ,&nbsp;Yanxia Liu ,&nbsp;Qingyao Wu ,&nbsp;F. Richard Yu","doi":"10.1016/j.neunet.2025.108011","DOIUrl":"10.1016/j.neunet.2025.108011","url":null,"abstract":"<div><div>Automatic segmentation of retinal vessels from retinography images is crucial for timely clinical diagnosis. However, the high cost and specialized expertise required for annotating medical images often result in limited labeled datasets, which constrains the full potential of deep learning methods. Recent advances in self-supervised pretraining using unlabeled data have shown significant benefits for downstream tasks. Recognizing that multimodal feature fusion can substantially enhance retinal vessel segmentation accuracy, this paper introduces a novel self-supervised pretraining framework that leverages pairs of unlabeled multimodal fundus images to generate supervisory signals. The core idea is to exploit the complementary differences between the two modalities to construct a multimodal feature fusion map containing vessel information, achieved through Vision Transformer encoding and correlation filtering. Instance-level discriminative features are then learned under the guidance of INFOMAX loss, and the learned knowledge is transferred to a supervised vessel segmentation network. Extensive experiments show that our approach achieves state-of-the-art results among unsupervised methods and remains competitive with supervised baselines while greatly reducing annotation requirements.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"193 ","pages":"Article 108011"},"PeriodicalIF":6.3,"publicationDate":"2025-09-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145010019","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Learning from history for personalized federated learning","authors":"Yingxun Fu,&nbsp;Shulan Yin,&nbsp;Li Ma,&nbsp;Jie Liu","doi":"10.1016/j.neunet.2025.108071","DOIUrl":"10.1016/j.neunet.2025.108071","url":null,"abstract":"<div><div>Personalized Federated Learning (pFL) has received extensive attentions, due to its ability to effectively process non-IID data distributed among different clients. However, most of the existing pFL methods focus on the collaboration between global and local models to enrich the personalization process, but ignoring a lot of valuable historical information, which represents the unique learning trajectory of each client. In this paper, we propose a pFL method called FedLFH, which introduces a tracking variable that allows each client to preserve historical information to facilitate personalization. We set up a global feature extractor and a personalized feature extractor for each client, to achieve the effective transfer of knowledge between the global model and the personalized model integrated with historical information. To evaluate the effectiveness, we set up exhaustive experiments on various benchmark datasets. The results show that our method outperforms twelve state-of-the-art methods with different experimental settings.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"193 ","pages":"Article 108071"},"PeriodicalIF":6.3,"publicationDate":"2025-09-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145010017","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Stabilization of delayed stochastic reaction-diffusion Cohen-Grossberg neural networks via variable gain intermittent boundary control","authors":"Yili Wang ,&nbsp;Wu-Hua Chen ,&nbsp;Shuning Niu ,&nbsp;Xiaoyun Lu","doi":"10.1016/j.neunet.2025.108041","DOIUrl":"10.1016/j.neunet.2025.108041","url":null,"abstract":"<div><div>This study presents a novel variable gain intermittent boundary control (VGIBC) approach for stabilizing delayed stochastic reaction-diffusion Cohen-Grossberg neural networks (SRDCGNN). In contrast to traditional constant gain intermittent boundary control (CGIBC) methods, the proposed VGIBC framework dynamically adjusts the control gain based on the operational duration within each control cycle, thereby improving adaptability to variations in work interval lengths. The time-varying control gain is designed using a piecewise interpolation method across work intervals, defined by a finite set of static gain matrices. To address the switching dynamics of the intermittently controlled neural networks and exploit the flexibility offered by the dynamic control gain, a piecewise Lyapunov function is employed to fit the dynamic structure of the control gain. By applying distinct Razumikhin-based solution estimation techniques: one tailored to active control periods and the other to rest periods, new mean square intermittent stabilization criteria are derived that show reduced conservatism compared to CGIBC-based results. The optimal control gain function is determined by solving a convex optimization procedure that minimizes the control rate at a given level of gain norm limitation. The efficacy of the proposed VGIBC strategy is validated through two numerical examples.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"193 ","pages":"Article 108041"},"PeriodicalIF":6.3,"publicationDate":"2025-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145010018","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A spatial-frequency hybrid restoration network for JPEG compressed image deblurring","authors":"Shu Tang ,&nbsp;Hanwen Zhang ,&nbsp;Xinbo Gao ,&nbsp;Shuli Yang ,&nbsp;Jiaxu Leng ,&nbsp;Zengdan Pan ,&nbsp;Hao Tian","doi":"10.1016/j.neunet.2025.108059","DOIUrl":"10.1016/j.neunet.2025.108059","url":null,"abstract":"<div><div>Image deblurring and compression-artifact removal are both ill-posed inverse problems in low-level vision tasks. So far, although numerous image deblurring and compression-artifact removal methods have been proposed respectively, the research for explicit handling blur and compression-artifact coexisting degradation image (BCDI) is rare. In the BCDI, image contents will be damaged more seriously, especially for edges and texture details. Therefore, the restoration of the BCDI is a more severe ill-posed inverse problem, and deep mining of local and global feature information is critical for effective BCDI restoration. To this end, we propose a spatial-frequency hybrid restoration network (SFHRN) for explicit and effective joint-photographic-experts-group (JPEG) compressed BCDI restoration. Specifically, according to the nature of JPEG compression artifacts, we propose a spatial-frequency hybrid block (SFHB), which includes a dual-branch structure and an information screening strategy (ISS). First, for the dual-branch structure, we design a patch-level channel attention branch (PCAB) and a pixel-level global attention branch (PGAB) to fully exploit local context information in the spatial domain and mine the global feature information in the frequency domain respectively. Secondly, we design a simple and effective information screening strategy (ISS) to discriminatively determine which pixels and channels should be retained and enhanced in frequency and spatial domains respectively for latent clear image restoration. Finally, for the first time, we build the blur and compression-artifact coexisting degradation datasets by adding various degrees of JPEG compression-artifact into existing benchmark deblurring datasets, e.g. GoPro and HIDE, named as GoPro-Compressed and HIDE-Compressed respectively. Extensive experiments demonstrate the superiority of our proposed SFHRN in terms of both performance and computational cost.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"193 ","pages":"Article 108059"},"PeriodicalIF":6.3,"publicationDate":"2025-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145010008","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Tangency portfolios using graph neural networks","authors":"Bin Liu ,&nbsp;Haolong Li ,&nbsp;Linshuang Kang","doi":"10.1016/j.neunet.2025.108043","DOIUrl":"10.1016/j.neunet.2025.108043","url":null,"abstract":"<div><div>According to modern portfolio theory, the weights of tangency portfolios are solely determined by the expected returns and the covariance matrix of asset returns. However, estimating expected returns and the covariance matrix poses significant challenges, especially when the number of assets is large. Considering the supply-demand relationships between companies issuing stocks, we propose that incorporating industry chain relationships can enhance the accurate estimation of the covariance matrix. Specifically, we present a method that employs Graph Neural Networks (GNNs) to estimate tangency portfolio weights by aggregating stock features based on the industry chain graph and using the aggregated features to estimate the expected returns and the covariance matrix. In addition to incorporating additional industry information, we propose two strategies to enhance the efficiency of estimation: 1) Calculating the dynamic modularity of the stock relationship graph using aggregated node features and constraining the estimated correlations to exhibit a clustered structure by minimizing modularity. 2) Adding a historical ranking regularization to the expected returns. We validate our approach on two daily stock datasets, demonstrating that our method effectively predicts portfolio returns and Sharpe ratios.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"193 ","pages":"Article 108043"},"PeriodicalIF":6.3,"publicationDate":"2025-08-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145027023","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}