Neural Networks最新文献

{"title":"Dynamic network compression via probabilistic channel pruning","authors":"Kwanhee Lee,&nbsp;Hyang-Won Lee","doi":"10.1016/j.neunet.2025.108080","DOIUrl":"10.1016/j.neunet.2025.108080","url":null,"abstract":"<div><div>Neural network compression problems have been extensively studied to overcome the limitations of compute-intensive deep learning models. Most of the state-of-the-art solutions in this context are based on network pruning that identify and remove unimportant weights, filters or channels. However, existing methods often lack actual speedup or require complex pruning criteria and additional training (fine-tuning) overhead. To address these limitations, we develop probability-based connectivity module that determines the connection of each channel to the next layer. Our connectivity module enables to dynamically activate and deactivate channel connections during training, and hence, does not necessitate fine-tuning of the pruned model. We show that the convolution decomposition, which decomposes convolution with connectivity module and depth-wise convolution can effectively induce sparsity, resulting in 52.76 %, 46.05 % reduction of parameter counts, with even boosting accuracy (+0.19 %, + 0.3 %) compared to baseline architectures in ResNet-56, VGG-19 Models. We also introduce resource-aware regularization that exploits the probabilistic activation of connectivity module in order to control the level of compression. We show that our method achieves comparable level of compression and accuracy to the state-of-the-art pruning methods.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"193 ","pages":"Article 108080"},"PeriodicalIF":6.3,"publicationDate":"2025-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145010021","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"WPDA: frequency-based backdoor attack with wavelet packet decomposition","authors":"Zhengyao Song ,&nbsp;Yongqiang Li ,&nbsp;Danni Yuan ,&nbsp;Li Liu ,&nbsp;Shaokui Wei ,&nbsp;Baoyuan Wu","doi":"10.1016/j.neunet.2025.108074","DOIUrl":"10.1016/j.neunet.2025.108074","url":null,"abstract":"<div><div>This work explores backdoor attack, which is an emerging security threat against deep neural networks (DNNs). The adversary aims to inject a backdoor into the model by manipulating a portion of training samples, such that the backdoor could be activated by a particular trigger to make a target prediction at inference. Currently, existing backdoor attacks often require moderate or high poisoning ratios to achieve the desired attack performance, but making them susceptible to some advanced backdoor defenses (<span><math><mrow><mi>e</mi><mo>.</mo><mi>g</mi><mo>.</mo></mrow></math></span>, poisoned sample detection). One possible solution to this dilemma is enhancing the attack performance at low poisoning ratios, which has been rarely studied due to its high challenge. To achieve this goal, we propose an innovative frequency-based backdoor attack via wavelet packet decomposition (WPD), which could finely decompose the original image into multiple sub-spectrograms with semantic information. It facilitates us to accurately identify the most critical frequency regions to effectively insert the trigger into the victim image, such that the trigger information could be sufficiently learned to form the backdoor. The proposed attack stands out for its exceptional effectiveness, stealthiness, and resistance at an extremely low poisoning ratio. Notably, it achieves the <span><math><mrow><mn>98.12</mn><mspace></mspace><mo>%</mo></mrow></math></span> attack success rate on CIFAR-10 with an extremely low poisoning ratio of <span><math><mrow><mn>0.004</mn><mspace></mspace><mo>%</mo></mrow></math></span> (<em>i.e.</em>, only 2 poisoned samples among 50,000 training samples), and bypasses several advanced backdoor defenses. Besides, we provide more extensive experiments to demonstrate the efficacy of the proposed method, as well as in-depth analyses to explain its underlying mechanism.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"194 ","pages":"Article 108074"},"PeriodicalIF":6.3,"publicationDate":"2025-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145082324","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A semi-looped functional for sampled-data synchronization of delayed neural networks considering communication delay","authors":"Yun-Hao An ,&nbsp;Xing-Chen Shangguan ,&nbsp;Hong-Zhang Wang ,&nbsp;Yu-Fei Peng ,&nbsp;Yun-Fan Liu ,&nbsp;Chuan-Ke Zhang","doi":"10.1016/j.neunet.2025.108076","DOIUrl":"10.1016/j.neunet.2025.108076","url":null,"abstract":"<div><div>This paper studies the master-slave synchronization of delayed neural networks (DNNs) using a sampled-data controller with a communication delay. First, a novel semi-looped functional is constructed to incorporate more system information and to feature more relaxed constraints, particularly the negative-definite condition on its derivatives. Second, two zero-value equations are constructed to fully coordinate the relationships among the system information introduced by the proposed functional, thereby providing greater flexibility in synchronization controller design. As a result, the synchronization criterion with reduced conservatism is derived by employing these techniques. This criterion allows for the design of a sampled-data synchronization controller for DNNs that accommodates larger sampling intervals, thus reducing communication and computational burdens. Finally, three widely used numerical examples illustrate the effectiveness and superiority of the proposed criterion.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"193 ","pages":"Article 108076"},"PeriodicalIF":6.3,"publicationDate":"2025-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145048479","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Dual aggregation based joint-modal similarity hashing for cross-modal retrieval","authors":"Le Xu,&nbsp;Jun Yin","doi":"10.1016/j.neunet.2025.108069","DOIUrl":"10.1016/j.neunet.2025.108069","url":null,"abstract":"<div><div>Cross-modal hashing aims to leverage hashing functions to map multimodal data into a unified low-dimensional space, realizing efficient cross-modal retrieval. In particular, unsupervised cross-modal hashing methods attract significant attention for not needing external label information. However, in the field of unsupervised cross-modal hashing, there are several pressing issues to address: (1) how to facilitate semantic alignment between modalities, and (2) how to effectively capture the intrinsic relationships between data, thereby constructing a more reliable affinity matrix to assist in the learning of hash codes. In this paper, Dual Aggregation-Based Joint-modal Similarity Hashing (DAJSH) is proposed to overcome these challenges. To enhance cross-modal semantic alignment, we employ a Transformer encoder to fuse image and text features and introduce a contrastive loss to optimize cross-modal consistency. Additionally, for constructing a more reliable affinity matrix to assist hash code learning, we propose a dual-aggregation affinity matrix construction scheme. This scheme integrates intra-modal cosine similarity and Euclidean distance while incorporating cross-modal similarity, thereby maximally preserving cross-modal semantic information. Experimental results demonstrate that our method achieves performance improvements of 1.9 % <span><math><mo>∼</mo></math></span> 5.1 %, 0.9 % <span><math><mo>∼</mo></math></span> 5.8 % and 0.6 % <span><math><mo>∼</mo></math></span> 2.6 % over state-of-the-art approaches on the MIR Flickr, NUS-WIDE and MS COCO benchmark datasets, respectively.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"193 ","pages":"Article 108069"},"PeriodicalIF":6.3,"publicationDate":"2025-09-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145010016","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Dormant key: Unlocking universal adversarial control in text-to-image models.","authors":"Jingqi Hu, Li Li, Hanzhou Wu, Huixin Luo, Xinpeng Zhang","doi":"10.1016/j.neunet.2025.108065","DOIUrl":"https://doi.org/10.1016/j.neunet.2025.108065","url":null,"abstract":"<p><p>Text-to-Image (T2I) diffusion models have gained significant traction due to their remarkable image generation capabilities, raising growing concerns over the security risks associated with their use. Prior studies have shown that malicious users can subtly modify prompts to produce visually misleading or Not-Safe-For-Work (NSFW) content, even bypassing existing safety filters. Existing adversarial attacks are often optimized for specific prompts, limiting their generalizability, and their text-space perturbations are easily detectable by current defenses. To address these limitations, we propose a universal adversarial attack framework called dormant key. It appends a transferable suffix that can be appended as a \"plug-in\" to any text input to guide the generated image toward a specific target. To ensure robustness across diverse prompts, we introduce a novel hierarchical gradient aggregation strategy that stabilizes optimization over prompt batches. This enables efficient learning of universal perturbations in the text space, improving both attack transferability and imperceptibility. Experimental results show that our method effectively balances attack performance and stealth. In NSFW generation tasks, it bypasses major safety mechanisms, including keyword filtering, semantic analysis, and text classifiers, and achieves over 18 % improvement in success rate over baselines.</p>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"193 ","pages":"108065"},"PeriodicalIF":6.3,"publicationDate":"2025-09-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145058622","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Hardware friendly deep reservoir computing","authors":"Claudio Gallicchio ,&nbsp;Miguel C. Soriano","doi":"10.1016/j.neunet.2025.108079","DOIUrl":"10.1016/j.neunet.2025.108079","url":null,"abstract":"<div><div>Reservoir Computing (RC) is a popular approach for modeling dynamical Recurrent Neural Networks, featured by a fixed (i.e., untrained) recurrent reservoir layer. In this paper, we introduce a novel design strategy for deep RC neural networks that is especially suitable to neuromorphic hardware implementations. From the topological perspective, the introduced model presents a multi-level architecture with ring reservoir topology and one-to-one inter-reservoir connections. The proposed design also considers hardware-friendly nonlinearity and noise modeling in the reservoir update equations. We demonstrate the introduced hardware-friendly deep RC architecture in electronic hardware, showing the promising processing capabilities on learning tasks that require both nonlinear computation and short-term memory. Additionally, we validate the effectiveness of the introduced approach on several time-series classification tasks, showing its competitive performance compared to its shallow counterpart, conventional, as well as more recent RC systems. These results emphasize the advantages of the proposed deep architecture for both practical hardware-friendly environments and broader machine learning applications.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"193 ","pages":"Article 108079"},"PeriodicalIF":6.3,"publicationDate":"2025-09-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145048483","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Large-margin Softmax loss using synthetic virtual class","authors":"Jiuzhou Chen ,&nbsp;Xiangyang Huang ,&nbsp;Shudong Zhang","doi":"10.1016/j.neunet.2025.108068","DOIUrl":"10.1016/j.neunet.2025.108068","url":null,"abstract":"<div><div>The primary challenge of large-margin learning lies in designing classifiers with strong discriminative power. Although existing large margin methods have achieved success in various classification tasks, they often suffer from weak task generalization and imbalanced handling of easy and hard samples. In this paper, we propose a margin adaptive synthetic virtual Softmax loss (SV-Softmax), which dynamically generates virtual prototypes by synthesizing embedded features and their corresponding prototypes. These virtual prototypes can adaptively adjust the margin based on the spatial distribution of embedded features, promoting the proximity of embedded features to their corresponding prototypes and creating clear and discriminative decision boundaries. Furthermore, we introduce a virtual prototype insertion strategy based on hard sample mining, where different synthesis strategies are applied to correctly and incorrectly classified samples, emphasizing the importance of hard samples. SV-Softmax is plug-and-play with minimal computational complexity, without requiring feature or weight normalization nor relying on task-specific hyperparameter tuning. Extensive comparative experiments on multiple visual classification and face recognition datasets demonstrate that SV-Softmax achieves competitive or superior performance compared to nine state-of-the-art methods. The code available at: <span><span>https://github.com/10zhou/SV-Softmax</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"193 ","pages":"Article 108068"},"PeriodicalIF":6.3,"publicationDate":"2025-09-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145019892","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Learning global-view correlation for salient object detection in 3D point clouds","authors":"Kan Huang ,&nbsp;Nannan Li ,&nbsp;Zhijing Xu","doi":"10.1016/j.neunet.2025.108078","DOIUrl":"10.1016/j.neunet.2025.108078","url":null,"abstract":"<div><div>Salient object detection (SOD) in point clouds has been an emerging research topic aimed at extracting most visually attractive objects from 3D point cloud representations. The inherent irregularity and unorderness of 3D point clouds complicate salient object detection, for it is hard to learn regular salient patterns like in 2D images. Meanwhile, existing methods typically focus on per-point context aggregation, while overlooking the scene-level global-view correlation crucial for saliency prediction. In this paper, we explore SOD in point clouds and introduce a novel approach that capitalizes on a comprehensive understanding of global-view 3D scenes. Our proposed method, the Saliency Filtration Network (SFN), meticulously refines saliency representations by isolating them from the common scene-dependent global-view correlations. Most importantly, SFN is characterized by a two-stage strategy, which involves aggregating long-range context information and purify saliency from globally scene-common correlations. To achieve this, we introduce the Residual Relation-aware Transformer module (RRT), which considers human visual perception to exploit global-view context dependencies. Additionally, we propose the Global Bilinear Correlation based Filtration module (GBCF) to perform saliency purification from global-view correlations. GBCF establishes dense correlations between global space and channel descriptors, which are then leveraged to properly purify saliency representations. Experimental evaluations on the PCSOD benchmark demonstrate that our proposed method achieves state-of-the-art accuracy and significantly outperforms other compared methods.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"193 ","pages":"Article 108078"},"PeriodicalIF":6.3,"publicationDate":"2025-09-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145048477","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Data-free knowledge distillation via text-noise fusion and dynamic adversarial temperature","authors":"Deheng Zeng ,&nbsp;Zhengyang Wu ,&nbsp;Yunwen Chen ,&nbsp;Zhenhua Huang","doi":"10.1016/j.neunet.2025.108061","DOIUrl":"10.1016/j.neunet.2025.108061","url":null,"abstract":"<div><div>Data-Free Knowledge Distillation (DFKD) have achieved significant breakthroughs, enabling the effective transfer of knowledge from teacher neural networks to student neural networks without reliance on original data. However, a significant challenge faced by existing methods that attempt to generate samples from random noise is that the noise lacks meaningful information, such as class-specific semantic information. Consequently, the absence of meaningful information makes it difficult for the generator to map this noise to the ground-truth data distribution, resulting in the generation of low-quality training samples. In addition, existing methods typically employ a fixed temperature for adversarial training of the generator, which limits the diversity in the difficulty of the synthesized data. In this paper, we propose Text-Noise Fusion and Dynamic Adversarial Temperature method (TNFDAT), a novel method that combines random noise with meaningful class-specific text embeddings (CSTE) as input and implements dynamic adjustment of the adversarial training temperature for the generator. In addition, we introduce an adaptive sample weighting strategy to enhance the effectiveness of knowledge distillation. CSTE is developed based on a pre-trained language model, and its significance lies in its ability to capture meaningful inter-class information, thereby enabling the generation of high-quality samples. Simultaneously, the dynamic adversarial temperature module effectively alleviates the issue of insufficient diversity in synthesized samples by precisely modulating the generator’s temperature during adversarial training, playing a key role in enhancing sample diversity. Through continuous and dynamic temperature adjustment of the generator in the adversarial training, thereby significantly improving the overall diversity of the synthesized samples. At the knowledge distillation stage, We determine the distillation weights of the synthesized samples based on the information entropy of the output from both teacher and student networks. By differentiating the contributions of different synthesized samples during the distillation process, we effectively enhance the generalization ability of the knowledge distillation framework and improve the robustness of the student network. Experiments demonstrate that our method outperforms the state-of-the-art methods across various benchmarks and pairs of teachers and students.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"193 ","pages":"Article 108061"},"PeriodicalIF":6.3,"publicationDate":"2025-09-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145019292","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Mitigating low-frequency bias: Feature recalibration and frequency attention regularization for adversarial robustness","authors":"Kejia Zhang ,&nbsp;Juanjuan Weng ,&nbsp;Yuanzheng Cai ,&nbsp;Shaozi Li ,&nbsp;Zhiming Luo","doi":"10.1016/j.neunet.2025.108070","DOIUrl":"10.1016/j.neunet.2025.108070","url":null,"abstract":"<div><div>Ensuring the robustness of deep neural networks against adversarial attacks remains a fundamental challenge in computer vision. While adversarial training (AT) has emerged as a promising defense strategy, our analysis reveals a critical limitation: AT-trained models exhibit a bias toward low-frequency features while neglecting high-frequency components. This bias is particularly concerning as each frequency component carries distinct and crucial information: low-frequency features encode fundamental structural patterns, while high-frequency features capture intricate details and textures. To address this limitation, we propose High-Frequency Feature Disentanglement and Recalibration (HFDR), a novel module that strategically separates and recalibrates frequency-specific features to capture latent semantic cues. We further introduce frequency attention regularization to harmonize feature extraction across the frequency spectrum and mitigate the inherent low-frequency bias of AT. Extensive experiments on CIFAR-10, CIFAR-100, and ImageNet-1K demonstrate that HFDR consistently enhances adversarial robustness. It achieves a 2.89 % gain on CIFAR-100 with WRN34-10, and improves robustness by 3.09 % on ImageNet-1K, with a 4.89 % gain on ViT-B against AutoAttack. These results highlight the method’s adaptability to both convolutional and transformer-based architectures. Code is available at <span><span>https://github.com/KejiaZhang-Robust/HFDR</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"193 ","pages":"Article 108070"},"PeriodicalIF":6.3,"publicationDate":"2025-09-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145027022","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}