Neural NetworksPub Date : 2026-07-01Epub Date: 2026-01-29DOI: 10.1016/j.neunet.2026.108657
Guhao Qiu , Ruoxin Chen , Zhihua Chen , Lei Dai , Ping Li , Bin Sheng
{"title":"Autorep: Automatic network search with structured reparameterized based linear operation expansion and gradient proxy guided reduction","authors":"Guhao Qiu , Ruoxin Chen , Zhihua Chen , Lei Dai , Ping Li , Bin Sheng","doi":"10.1016/j.neunet.2026.108657","DOIUrl":"10.1016/j.neunet.2026.108657","url":null,"abstract":"<div><div>Convolution neural network and Vision Transformer have achieved large success in various computer vision tasks. However, the huge computation cost hinders its application and it is hard to design efficient methods to obtain lightweight architectures with both mannually designed strategies and automatically searching methods. In this paper, we focus on introducing the specific structural reparameterization strategy in SuperNet training to improve the performance of one-shot based neural architecture search algorithm. During the SuperNet training process, each candidate operation is expanded by a series of equivalent operation branches to fully utilize the representation potential. To alleviate the training difficulty and avoid bringing too much computation costs, the operation reduction strategy and prior sampling strategy are used after validating the sampled subnetworks. The operation reduction strategy is to remove the low-effect extended linear layer. The reduction step needs to firstly select the candidate operation based on SynFlow proxy and then select the extended linear layer from the selected operation based on the accuracy difference before and after removal.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"199 ","pages":"Article 108657"},"PeriodicalIF":6.3,"publicationDate":"2026-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146174713","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"CGE-GAN: Contrastive-guided evolutionary generative adversarial networks with dynamic adaptive weight sharing","authors":"Kashif Iqbal , Xue Yu , Atifa Rafique , Muhammad Hamid , Khursheed Aurangzeb","doi":"10.1016/j.neunet.2026.108702","DOIUrl":"10.1016/j.neunet.2026.108702","url":null,"abstract":"<div><div>Generative adversarial networks (GANs) have achieved remarkable success in image synthesis but faces major challenges, including mode collapse, training instability, and inefficient architecture search. Existing evolutionary GANs partially address these issues but lack semantic alignment with real data, effective weight reuse, and knowledge transfer between model generations. To overcome these limitations, we propose contrastive-guided evolutionary GANs (CGE-GAN)–a unified method introduces a novel hybrid Wasserstein-Contrastive loss function that drives generators to align semantically with real data while maintaining adversarial competitiveness. Besides, we incorporated dynamic adaptive weight sharing (DAWS) for efficient training and knowledge distillation-based crossover to preserve useful features across generations. The CGE-GAN is evaluated on CIFAR-10 and STL-10, and it achieves an Inception Score (IS) of 8.99 and 10.46, and fréchet inception distance (FID) of 9.74 and 21.86, respectively. Compared to strong baselines, CGE-GAN reduces FID by up to 1.74 points while maintaining high semantic diversity and convergence efficiency with only 0.36 GPU days. These results highlight the effectiveness of contrastive-driven evolution for generating stable and high-fidelity outputs.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"199 ","pages":"Article 108702"},"PeriodicalIF":6.3,"publicationDate":"2026-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146174718","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Neural NetworksPub Date : 2026-07-01Epub Date: 2026-01-29DOI: 10.1016/j.neunet.2026.108658
Shuda Zhang , Huiying Li
{"title":"Hierarchical ranking in hyperbolic space: A novel approach to metric learning","authors":"Shuda Zhang , Huiying Li","doi":"10.1016/j.neunet.2026.108658","DOIUrl":"10.1016/j.neunet.2026.108658","url":null,"abstract":"<div><div>The integration of deep metric learning with hyperbolic geometry has shown significant potential for capturing complex hierarchical relationships. However, existing clustering-based methods struggle to fully leverage the properties of hyperbolic space, particularly due to the challenge of optimizing both cluster centers and distance metrics in exponentially expanding spaces without true hierarchical labels. Additionally, the computational complexity of Riemannian operations makes maintaining hierarchical structures costly, especially for large datasets. To address these challenges, we propose a novel hierarchical ranking framework that utilizes latent hierarchical information without relying on explicit clustering. This framework introduces the Hierarchical Ranking Generation (HRG) strategy and Hierarchical Ranking Loss (HRL). HRG generates ranking labels based on the semantic relationships between classes within an implicit global hierarchy, while HRL optimizes these rankings across multiple hierarchical levels, enabling the model to learn richer, more nuanced representations. Our approach significantly improves performance, outperforming the state-of-the-art by 2.4% on CUB-200-2011 and 1.6% on Cars-196 at Recall@1.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"199 ","pages":"Article 108658"},"PeriodicalIF":6.3,"publicationDate":"2026-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146174720","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Neural NetworksPub Date : 2026-07-01Epub Date: 2026-01-29DOI: 10.1016/j.neunet.2026.108649
Mengjuan Jiang, Fanzhang Li
{"title":"Learning discriminative prototypes: Adaptive relation-aware refinement and patch-level contextual feature reweighting for few-shot classification","authors":"Mengjuan Jiang, Fanzhang Li","doi":"10.1016/j.neunet.2026.108649","DOIUrl":"10.1016/j.neunet.2026.108649","url":null,"abstract":"<div><div>Few-shot learning (FSL) aims to achieve efficient classification with limited labeled samples, providing an important research paradigm for addressing the model generalization issue in data-scarce scenarios. In the metric-based FSL framework, class prototypes serve as the core transferable representation of classes, and their discriminative power directly impacts the model’s classification performance. However, existing methods face two major bottlenecks: first, traditional feature selection mechanisms use static modeling approaches that are susceptible to background noise and struggle to capture dynamic relationships between classes; second, due to limitations in the quantity and quality of labeled samples, prototype representations based on global features lack fine-grained expression of local discriminative features, limiting the prototype’s representational power. To overcome these limitations, we propose a novel framework: Learning Discriminative Prototypes (LDP). LDP includes two modules: (1) Adaptive relation-aware refinement, which dynamically models the relationships between class prototypes, highlighting the key features of each class and effectively enhancing the robustness of feature representations; (2) Patch-level contextual feature reweighting, which performs a reweighting operation on the samples through patch-level feature interactions thereby obtaining a more discriminative prototype. Experimental results demonstrate that LDP shows strong competitiveness on five datasets covering both standard and cross-domain datasets, validating its effectiveness in FSL tasks. For example, in the 1-shot setting on miniImageNet and tieredImageNet, LDP achieves over 12% accuracy improvement compared with the baseline methods; on the cross-domain dataset CUB200, the improvement reaches 6.45% in the 1-shot case. Our code is available on GitHub at <span><span>https://github.com/fewshot-learner/LDP</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"199 ","pages":"Article 108649"},"PeriodicalIF":6.3,"publicationDate":"2026-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146120781","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Neural NetworksPub Date : 2026-07-01Epub Date: 2026-02-03DOI: 10.1016/j.neunet.2026.108680
Ben Liang , Yuan Liu , Chao Sui , Yihong Wang , Lin Xiao , Xiubao Sui , Qian Chen
{"title":"Multi-Scale pattern-Aware task-Gating network for aerial small object detection","authors":"Ben Liang , Yuan Liu , Chao Sui , Yihong Wang , Lin Xiao , Xiubao Sui , Qian Chen","doi":"10.1016/j.neunet.2026.108680","DOIUrl":"10.1016/j.neunet.2026.108680","url":null,"abstract":"<div><div>With the advancement of high-precision remote sensing equipment and precision measurement technology, object detection based on remote sensing images (RSIs) has been widely used in military and civilian fields. Different from traditional general-purpose environments, remote sensing presents unique challenges that significantly complicate the detection process. Specifically: (1) RSIs cover extensive monitoring areas, resulting in complex and textured backgrounds; and (2) objects often exhibit cluttered distributions, small sizes, and considerable scale variations across categories. To effectively address these challenges, we propose a Multi-Scale Pattern-Aware Task-Gating Network (MPTNet) for remote sensing object detection. First, we design a Multi-Scale Pattern-Aware Network (MPNet) backbone that employs a small and large kernel convolutional complementary strategy to capture both large-scale and small-scale spatial patterns, yielding more comprehensive semantic features. Next, we introduce a Multi-Head Cross-Space Encoder (MCE) that improves semantic fusion and spatial representation across hierarchical levels. By combining a multi-head mechanism with directional one-dimensional strip convolutions, MCE enhances spatial sensitivity at the pixel level, thus improving object localization in densely textured scenes. To harmonize cross-task synergy, we propose a Dynamic Task-Gating (DTG) head that adaptively recalibrates spatial feature representations between classification and localization branches. Extensive experimental validations on three publicly available datasets, including VisDrone, DIOR, and COCO-mini, demonstrate that our method achieves excellent performance, obtaining AP<sub>50</sub> scores of 43.3%, 80.6%, and 49.5%, respectively.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"199 ","pages":"Article 108680"},"PeriodicalIF":6.3,"publicationDate":"2026-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146144671","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Neural NetworksPub Date : 2026-07-01Epub Date: 2026-02-02DOI: 10.1016/j.neunet.2026.108667
Gong Gao, Weidong Zhao, Xianhui Liu, Ning Jia
{"title":"Improving policy exploitation in online reinforcement learning with instant retrospect action","authors":"Gong Gao, Weidong Zhao, Xianhui Liu, Ning Jia","doi":"10.1016/j.neunet.2026.108667","DOIUrl":"10.1016/j.neunet.2026.108667","url":null,"abstract":"<div><div>Existing value-based online reinforcement learning (RL) algorithms suffer from slow policy exploitation due to ineffective exploration and delayed policy updates. To address these challenges, we propose an algorithm called Instant Retrospect Action (IRA). Specifically, we propose Q-Representation Discrepancy Evolution (RDE) to facilitate Q-network representation learning, enabling discriminative representations for neighboring state-action pairs. In addition, we adopt an explicit method to policy constraints by enabling Greedy Action Guidance (GAG). This is achieved through backtracking historical actions, which effectively enhances the policy update process. Our proposed method relies on providing the learning algorithm with accurate <em>k</em>-nearest-neighbor action value estimates and learning to design a fast-adaptable policy through policy constraints. We further propose the Instant Policy Update (IPU) mechanism, which enhances policy exploitation by systematically increasing the frequency of policy updates. We further discover that the early-stage training conservatism of the IRA method can alleviate the overestimation bias problem in value-based RL. Experimental results show that IRA can significantly improve the learning efficiency and final performance of online RL algorithms on eight MuJoCo continuous control tasks.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"199 ","pages":"Article 108667"},"PeriodicalIF":6.3,"publicationDate":"2026-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146158658","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Neural NetworksPub Date : 2026-07-01Epub Date: 2026-01-24DOI: 10.1016/j.neunet.2026.108641
Georg Kohl, Li-Wei Chen, Nils Thuerey
{"title":"Benchmarking autoregressive conditional diffusion models for turbulent flow simulation","authors":"Georg Kohl, Li-Wei Chen, Nils Thuerey","doi":"10.1016/j.neunet.2026.108641","DOIUrl":"10.1016/j.neunet.2026.108641","url":null,"abstract":"<div><div>Simulating turbulent flows is crucial for a wide range of applications, and machine learning-based solvers are gaining increasing relevance. However, achieving temporal stability when generalizing to longer rollout horizons remains a persistent challenge for learned PDE solvers. In this work, we analyze if fully data-driven fluid solvers that utilize an autoregressive rollout based on conditional diffusion models are a viable option to address this challenge. We investigate accuracy, posterior sampling, spectral behavior, and temporal stability, while requiring that methods generalize to flow parameters beyond the training regime. To quantitatively and qualitatively benchmark the performance of various flow prediction approaches, three challenging 2D scenarios including incompressible and transonic flows, as well as isotropic turbulence are employed. We find that even simple diffusion-based approaches can outperform multiple established flow prediction methods in terms of accuracy and temporal stability, while being on par with state-of-the-art stabilization techniques like unrolling at training time. Such traditional architectures are superior in terms of inference speed, however, the probabilistic nature of diffusion approaches allows for inferring multiple predictions that align with the statistics of the underlying physics. Overall, our benchmark contains three carefully chosen data sets that are suitable for probabilistic evaluation alongside various established flow prediction architectures.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"199 ","pages":"Article 108641"},"PeriodicalIF":6.3,"publicationDate":"2026-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"146151141","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Neural NetworksPub Date : 2026-06-01Epub Date: 2026-01-07DOI: 10.1016/j.neunet.2026.108539
Lingxiao Qu , Yan Pei
{"title":"Kernelized linear principal component discriminant analysis","authors":"Lingxiao Qu , Yan Pei","doi":"10.1016/j.neunet.2026.108539","DOIUrl":"10.1016/j.neunet.2026.108539","url":null,"abstract":"<div><div>In this paper, we propose Kernelized Linear Principal Component Discriminant Analysis (KLPCDA), a structured and unified framework for discriminant analysis that overcomes the fragmentation in existing multi-stage approaches such as PCA+LDA or KPCA+GDA. Instead of treating feature extraction and class discrimination as disjoint steps, KLPCDA formulates a joint optimization model in the Reproducing Kernel Hilbert Space (RKHS), integrating overall variance preservation, between-class separation, and within-class compactness into a fused objective. The formulation supports seven KLPCDA variants, offering flexible control over effects of each criterion through tunable fusion coefficients. We present a systematic parameter optimization strategy, including kernel parameter selection, subspace dimensionality tuning, and fusion balancing, along with an alternative kernel parameter optimization method. Extensive experiments across image, tabular, and signal datasets across small-sample-size (SSS) to larger-scale settings validate the adaptability of KLPCDA. The results demonstrate KLPCDA consistently outperforms benchmark methods and convolutional neural networks in SSS settings on both recognition accuracy and efficiency, while maintaining competitive advantages in computational complexity and storage requirements for large-scale scenarios. Finally, we provide insights into extended study subjects and future work related to our proposal.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"198 ","pages":"Article 108539"},"PeriodicalIF":6.3,"publicationDate":"2026-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145967199","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Neural NetworksPub Date : 2026-06-01Epub Date: 2026-01-10DOI: 10.1016/j.neunet.2026.108573
Xuchao Liu, Qi Cao, Kaike Zhang, Du Su, Huawei Shen
{"title":"The answer lies within: Detecting Trojans from DNNs’ inherent characteristics","authors":"Xuchao Liu, Qi Cao, Kaike Zhang, Du Su, Huawei Shen","doi":"10.1016/j.neunet.2026.108573","DOIUrl":"10.1016/j.neunet.2026.108573","url":null,"abstract":"<div><div>Deep neural networks (DNNs) are vulnerable to Trojan attacks, where adversaries implant Trojans that cause DNNs to misbehave when encountering specific triggers. Detecting Trojans in DNNs is crucial to mitigate potential safety risks. Traditional methods typically employ trigger reversion techniques, which utilize benign samples to reconstruct potential triggers through iterative optimization. However, their practical applicability is limited by reliance on benign samples and the exceedingly time-intensive optimization. In this paper, we investigate a more general yet challenging setting, the benign sample-free scenario, where detection relies solely on DNN itself. We propose a novel approach for <strong><u>d</u></strong>etecting <strong><u>T</u></strong>rojans from DNNs’ <strong><u>i</u></strong>nherent <strong><u>c</u></strong>haracteristics (DTIC), which exploits the distinguishable features of Trojaned models. DTIC depicts the characteristics of various DNNs via a unified representation space derived from both views of model structures and parameters, enabling adaptability across diverse DNNs. It requires just one direct inference to assess the presence of Trojans, ensuring high efficiency. We further enhance the performance of Trojan detection, using augmentations based on random perturbations and the lottery hypothesis. Extensive experiments conducted on IARPA TrajAI<span><span><sup>1</sup></span></span>, a widely adopted benchmark, demonstrate the superior effectiveness, efficiency, and generalizability of DTIC.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"198 ","pages":"Article 108573"},"PeriodicalIF":6.3,"publicationDate":"2026-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145980477","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Neural NetworksPub Date : 2026-06-01Epub Date: 2026-01-11DOI: 10.1016/j.neunet.2026.108543
Paolo Conti , Jonas Kneifl , Andrea Manzoni , Attilio Frangi , Jörg Fehr , Steven L. Brunton , J. Nathan Kutz
{"title":"VENI, VINDy, VICI: A generative reduced-order modeling framework with uncertainty quantification","authors":"Paolo Conti , Jonas Kneifl , Andrea Manzoni , Attilio Frangi , Jörg Fehr , Steven L. Brunton , J. Nathan Kutz","doi":"10.1016/j.neunet.2026.108543","DOIUrl":"10.1016/j.neunet.2026.108543","url":null,"abstract":"<div><div>Generative models are transforming science and engineering by enabling efficient synthetization and exploration of new scenarios for complex physical phenomena with minimal cost. Although they provide uncertainty-aware predictions to support decision making, they typically lack physical consistency, which is the backbone of computational science. Hence, we propose VENI, VINDy, VICI – a novel <em>physical generative framework</em> that integrates data-driven system identification into a probabilistic modeling approach to construct physically consistent and efficient reduced-order models with uncertainty quantification. First, VENI (Variational Encoding of Noisy Inputs) employs variational autoencoders to identify reduced coordinates from high-dimensional, noisy measurements. Simultaneously, VINDy (Variational Identification of Nonlinear Dynamics) extends sparse system identification methods by embedding probabilistic modeling into the discovery process. Last, VICI (Variational Inference with Credibility Intervals) enables efficient generation of full-time solutions and provides uncertainty quantification for unseen parameters and initial conditions. We demonstrate the performance of the framework across chaotic and high-dimensional nonlinear systems.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"198 ","pages":"Article 108543"},"PeriodicalIF":6.3,"publicationDate":"2026-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145999302","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}