Neural Networks最新文献_第4页

Adaptive node-level weighted learning for directed graph neural network

IF 6 1区计算机科学

Neural Networks Pub Date : 2025-03-21 DOI: 10.1016/j.neunet.2025.107393

Jincheng Huang , Xiaofeng Zhu

引用次数: 0

More signals matter to detection: Integrating language knowledge and frequency representations for boosting fine-grained aircraft recognition

IF 6 1区计算机科学

Neural Networks Pub Date : 2025-03-21 DOI: 10.1016/j.neunet.2025.107402

Xueru Xu, Zhong Chen, Yuxin Hu, Guoyou Wang

{"title":"More signals matter to detection: Integrating language knowledge and frequency representations for boosting fine-grained aircraft recognition","authors":"Xueru Xu, Zhong Chen, Yuxin Hu, Guoyou Wang","doi":"10.1016/j.neunet.2025.107402","DOIUrl":"10.1016/j.neunet.2025.107402","url":null,"abstract":"<div><div>As object detection tasks progress rapidly, fine-grained detection flourishes as a promising extension. Fine-grained recognition naturally demands high-quality detail signals; however, existing fine-grained detectors, built upon the mainstream detection paradigm, struggle to simultaneously address the challenges of insufficient original signals and the loss of critical signals, resulting in inferior performance. We argue that language signals with advanced semantic knowledge can provide valuable information for fine-grained objects, as well as the frequency domain exhibits greater flexibility in suppressing and enhancing signals; then, we propose a fine-grained aircraft detector by integrating language knowledge and frequency representations into the one-stage detection paradigm. Concretely, by considering both original signals and deep feature signals, we develop three components, including an adaptive frequency augmentation branch (AFAB), a content-aware global features intensifier (CGFI), and a fine-grained text–image interactive feeder (FTIF), to facilitate perceiving and retaining critical signals throughout pivotal detection stages. The AFAB adaptively processes image patches according to their frequency characteristics in the Fourier domain, thus thoroughly mining critical visual content in the data space; the CGFI employs content-aware frequency filtering to enhance global features, allowing for generating an information-rich feature space; the FTIF introduces text knowledge to describe visual differences among fine-grained categories, conveying robust semantic priors from language signals to visual spaces via multimodal interaction for information supplement. Extensive experiments conducted on optical and SAR images demonstrate the superior performance of the proposed fine-grained detector, especially the FTIF, which can be plugged into most existing one-stage detectors to boost their fine-grained recognition performance significantly.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"187 ","pages":"Article 107402"},"PeriodicalIF":6.0,"publicationDate":"2025-03-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143679980","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Intra-class progressive and adaptive self-distillation

IF 6 1区计算机科学

Neural Networks Pub Date : 2025-03-21 DOI: 10.1016/j.neunet.2025.107404

Jianping Gou , Jiaye Lin , Lin Li , Weihua Ou , Baosheng Yu , Zhang Yi

{"title":"Intra-class progressive and adaptive self-distillation","authors":"Jianping Gou , Jiaye Lin , Lin Li , Weihua Ou , Baosheng Yu , Zhang Yi","doi":"10.1016/j.neunet.2025.107404","DOIUrl":"10.1016/j.neunet.2025.107404","url":null,"abstract":"<div><div>In recent years, knowledge distillation (KD) has become widely used in compressing models, training compact and efficient students to reduce computational load and training time due to the increasing parameters in deep neural networks. To minimize training costs, self-distillation has been proposed, with methods like offline-KD and online-KD requiring pre-trained teachers and multiple networks. However, these self-distillation methods often overlook feature knowledge and category information. In this paper, we introduce Intra-class Progressive and Adaptive Self-Distillation (IPASD), which transfers knowledge from the front to the back in adjacent epochs. This method extracts class-typical features and promotes compactness within classes. By integrating feature-level and logits-level knowledge into strong teacher knowledge and using ground-truth labels as supervision signals, we adaptively optimize the model. We evaluated IPASD on CIFAR-10, CIFAR-100, Tiny ImageNet, Plant Village datasets, and ImageNet showing its superiority over state-of-the-art self-distillation methods in knowledge transfer and model compression. Our codes are available at: <span><span>https://github.com/JLinye/IPASD</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"188 ","pages":"Article 107404"},"PeriodicalIF":6.0,"publicationDate":"2025-03-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143704893","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Structure information preserving domain adaptation network for fault diagnosis of Sucker Rod Pumping systems

IF 6 1区计算机科学

Neural Networks Pub Date : 2025-03-21 DOI: 10.1016/j.neunet.2025.107392

Xiaohua Gu , Fei Lu , Liping Yang , Kan Wang , Lusi Li , Guang Yang , Yiling Sun

{"title":"Structure information preserving domain adaptation network for fault diagnosis of Sucker Rod Pumping systems","authors":"Xiaohua Gu , Fei Lu , Liping Yang , Kan Wang , Lusi Li , Guang Yang , Yiling Sun","doi":"10.1016/j.neunet.2025.107392","DOIUrl":"10.1016/j.neunet.2025.107392","url":null,"abstract":"<div><div>Fault diagnosis is of great importance to the reliability and security of Sucker Rod Pumping (SRP) oil production system. With the development of digital oilfield, data-driven deep learning SRP fault diagnosis has become the development trend of oilfield system. However, due to the different working conditions, time periods, and areas, the fault diagnosis models trained from certain SRP data do not consider the statistical discrepancy of different SRP systems, resulting in insufficient generalization. To consider the fault diagnosis and generalization performances of deep models at the same time, this paper proposes a Structure Information Preserving Domain Adaptation Network (SIP-DAN) for SRP fault diagnosis. Different from the usual domain adaptation methods, SIP-DAN divides the source domain data into different subdomains according to the fault categories of the source domain, and then realizes structure information preserving domain adaptation through subdomains alignment of the source domain and the target domain. Due to the lack of fault category information in the target domain, we designed a Classifier Voting Assisted Alignment (CVAA) mechanism. The target domain data are divided into clusters using fuzzy clustering algorithm. Then, fault diagnosis classifier trained in source domain is employed to classify the samples in each cluster, and the majority voting principle is used to assign pseudo-labels to each cluster in the target domain. With these pseudo-labels, source and target subdomains alignment is carried out by optimizing the Local Maximum Mean Discrepancy (LMMD) loss to achieve fine-grained domain adaptation. Experimental results illustrate that the proposed method is better than the existing methods in fault diagnosis of SRP systems.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"188 ","pages":"Article 107392"},"PeriodicalIF":6.0,"publicationDate":"2025-03-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143714782","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Central loss guides coordinated Transformer for reliable anatomical landmark detection

IF 6 1区计算机科学

Neural Networks Pub Date : 2025-03-21 DOI: 10.1016/j.neunet.2025.107391

Qikui Zhu , Yihui Bi , Jie Chen , Xiangpeng Chu , Danxin Wang , Yanqing Wang

{"title":"Central loss guides coordinated Transformer for reliable anatomical landmark detection","authors":"Qikui Zhu , Yihui Bi , Jie Chen , Xiangpeng Chu , Danxin Wang , Yanqing Wang","doi":"10.1016/j.neunet.2025.107391","DOIUrl":"10.1016/j.neunet.2025.107391","url":null,"abstract":"<div><div>Heatmap-based anatomical landmark detection is still facing two unresolved challenges: (1) inability to accurately evaluate the distribution of heatmap; (2) inability to effectively exploit global spatial structure information. To address the computational inability challenge, we propose a novel position-aware and sample-aware central loss. Specifically, our central loss can absorb position information, enabling accurate evaluation of the heatmap distribution. More advanced is that our central loss is sample-aware, which can adaptively distinguish easy and hard samples and make the model more focused on hard samples while solving the challenge of extreme imbalance between landmarks and non-landmarks. To address the challenge of ignoring structure information, a Coordinated Transformer, called CoorTransformer, is proposed, which establishes long-range dependencies under the guidance of landmark coordinate information, making the attention more focused on the sparse landmarks while taking advantage of global spatial structure. Furthermore, CoorTransformer can speed up convergence, effectively avoiding the defect that Transformers have difficulty converging in sparse representation learning. Using the advanced CoorTransformer and central loss, we propose a generalized detection model that can handle various scenarios, inherently exploiting the underlying relationship between landmarks and incorporating rich structural knowledge around the target landmarks. We analyzed and evaluated CoorTransformer and central loss on three challenging landmark detection tasks. The experimental results show that our CoorTransformer outperforms state-of-the-art methods, and the central loss significantly improves the model’s performance with <span><math><mi>p</mi></math></span>-values <span><math><mrow><mo><</mo><mn>0</mn><mo>.</mo><mn>05</mn></mrow></math></span>. The source code of this work is available at the <span><span>GitHub repository</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"187 ","pages":"Article 107391"},"PeriodicalIF":6.0,"publicationDate":"2025-03-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143696808","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Mathematical expression exploration with graph representation and generative graph neural network

IF 6 1区计算机科学

Neural Networks Pub Date : 2025-03-21 DOI: 10.1016/j.neunet.2025.107405

Jingyi Liu , Weijun Li , Lina Yu , Min Wu , Wenqiang Li , Yanjie Li , Meilan Hao

{"title":"Mathematical expression exploration with graph representation and generative graph neural network","authors":"Jingyi Liu , Weijun Li , Lina Yu , Min Wu , Wenqiang Li , Yanjie Li , Meilan Hao","doi":"10.1016/j.neunet.2025.107405","DOIUrl":"10.1016/j.neunet.2025.107405","url":null,"abstract":"<div><div>Symbolic Regression (SR) methods in tree representations have exhibited commendable outcomes across Genetic Programming (GP) and deep learning search paradigms. Nonetheless, the tree representation of mathematical expressions occasionally embodies redundant substructures. Representing expressions as computation graphs is more succinct and intuitive through graph representation. Despite its adoption in evolutionary strategies within SR, deep learning paradigms remain under-explored. Acknowledging the profound advancements of deep learning in tree-centric SR approaches, we advocate for addressing SR tasks using the Directed Acyclic Graph (DAG) representation of mathematical expressions, complemented by a generative graph neural network. We name the proposed method as <em><strong>Graph</strong>-based <strong>D</strong>eep <strong>S</strong>ymbolic <strong>R</strong>egression (GraphDSR)</em>. We vectorize node types and employ an adjacent matrix to delineate connections. The graph neural networks craft the DAG incrementally, sampling node types and graph connections conditioned on previous DAG at every step. During each sample step, the valid check is implemented to avoid meaningless sampling, and four domain-agnostic constraints are adopted to further streamline the search. This process culminates once a coherent expression emerges. Constants undergo optimization by SGD and BFGS algorithms, and rewards refine the graph neural network through reinforcement learning. A comprehensive evaluation across 110 benchmarks underscores the potency of our approach.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"187 ","pages":"Article 107405"},"PeriodicalIF":6.0,"publicationDate":"2025-03-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143726271","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Enabling scale and rotation invariance in convolutional neural networks with retina like transformation

IF 6 1区计算机科学

Neural Networks Pub Date : 2025-03-20 DOI: 10.1016/j.neunet.2025.107395

Jiahong Zhang , Guoqi Li , Qiaoyi Su , Lihong Cao , Yonghong Tian , Bo Xu

{"title":"Enabling scale and rotation invariance in convolutional neural networks with retina like transformation","authors":"Jiahong Zhang , Guoqi Li , Qiaoyi Su , Lihong Cao , Yonghong Tian , Bo Xu","doi":"10.1016/j.neunet.2025.107395","DOIUrl":"10.1016/j.neunet.2025.107395","url":null,"abstract":"<div><div>Traditional convolutional neural networks (CNNs) struggle with scale and rotation transformations, resulting in reduced performance on transformed images. Previous research focused on designing specific CNN modules to extract transformation-invariant features. However, these methods lack versatility and are not adaptable to a wide range of scenarios. Drawing inspiration from human visual invariance, we propose a novel brain-inspired approach to tackle the invariance problem in CNNs. If we consider a CNN as the visual cortex, we have the potential to design an “eye” that exhibits transformation invariance, allowing CNNs to perceive the world consistently. Therefore, we propose a retina module and then integrate it into CNNs to create transformation-invariant CNNs (TICNN), achieving scale and rotation invariance. The retina module comprises a retina-like transformation and a transformation-aware neural network (TANN). The retina-like transformation supports flexible image transformations, while the TANN regulates these transformations for scaling and rotation. Specifically, we propose a reference-based training method (RBTM) where the retina module learns to align input images with a reference scale and rotation, thereby achieving invariance. Furthermore, we provide mathematical substantiation for the retina module to confirm its feasibility. Experimental results also demonstrate that our method outperforms existing methods in recognizing images with scale and rotation variations. The code will be released at <span><span>https://github.com/JiaHongZ/TICNN</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"187 ","pages":"Article 107395"},"PeriodicalIF":6.0,"publicationDate":"2025-03-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143679904","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Feature-Tuning Hierarchical Transformer via token communication and sample aggregation constraint for object re-identification

IF 6 1区计算机科学

Neural Networks Pub Date : 2025-03-20 DOI: 10.1016/j.neunet.2025.107394

Zhi Yu , Zhiyong Huang , Mingyang Hou , Jiaming Pei , Yan Yan , Yushi Liu , Daming Sun

{"title":"Feature-Tuning Hierarchical Transformer via token communication and sample aggregation constraint for object re-identification","authors":"Zhi Yu , Zhiyong Huang , Mingyang Hou , Jiaming Pei , Yan Yan , Yushi Liu , Daming Sun","doi":"10.1016/j.neunet.2025.107394","DOIUrl":"10.1016/j.neunet.2025.107394","url":null,"abstract":"<div><div>Recently, transformer-based methods have shown remarkable success in object re-identification. However, most works directly embed off-the-shelf transformer backbones for feature extraction. These methods treat all patch tokens equally, ignoring the difference of distinct patch tokens for feature representation. To solve this issue, this paper designs a feature-tuning mechanism for transformer backbones to emphasize important patches and attenuate unimportant patches. Specifically, a Feature-tuning Hierarchical Transformer (FHTrans) for object re-identification is proposed. First, we propose a plug-and-play Feature-tuning module via Token Communication (TCF) deployed within transformer encoder blocks. This module regards the class token as a pivot to achieve communication between patch tokens. Important patch tokens are emphasized, while unimportant patch tokens are attenuated, focusing more precisely on the discriminative features related to object distinction. Then, we construct a FHTrans based on the designed feature-tuning module. The encoder blocks are divided into three hierarchies considering the correlation between feature representativeness and transformer depth. As the hierarchy deepens, the communication between tokens becomes tighter. This enables the model to capture more crucial feature information. Finally, we propose a Sample Aggregation (SA) loss to impose more effective constraints on statistical characteristics among samples, thereby enhancing intra-class aggregation and guiding FHTrans to learn more discriminative features. Experiments on object re-identification benchmarks demonstrate that our method can achieve state-of-the-art performance.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"187 ","pages":"Article 107394"},"PeriodicalIF":6.0,"publicationDate":"2025-03-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143679899","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Estimating uncertainty from feed-forward network based sensing using quasi-linear approximation

IF 6 1区计算机科学

Neural Networks Pub Date : 2025-03-20 DOI: 10.1016/j.neunet.2025.107376

Songhan Zhang , Matthew Singh , Delsin Menolascino , ShiNung Ching

{"title":"Estimating uncertainty from feed-forward network based sensing using quasi-linear approximation","authors":"Songhan Zhang , Matthew Singh , Delsin Menolascino , ShiNung Ching","doi":"10.1016/j.neunet.2025.107376","DOIUrl":"10.1016/j.neunet.2025.107376","url":null,"abstract":"<div><div>A fundamental problem in neural network theory is the quantification of uncertainty as it propagates through these constructs. Such quantification is crucial as neural networks become integrated into broader engineered systems that render decisions based on their outputs. In this paper, we engage the problem of estimating uncertainty in feedforward neural network constructs. Mathematically, the problem, in essence, amounts to understanding how the moments of an input distribution become modifies as they move through network layers. Despite its straightforward formulation, the nonlinear nature of modern feedforward architectures makes this is a mathematically challenging problem. Most contemporary approaches rely on some form of Monte Carlo sampling to construct inter-laminar distributions. Here, we borrow an approach from the control systems community known as quasilinear approximation, to enable a more analytical approach to the uncertainty quantification problem in this setting. Specifically, by using quasilinear approximation, nonlinearities are linearized in terms of the expectation of their gain in an input–output sense. We derive these expectations for several commonly used nonlinearities, under the assumption of Gaussian inputs. We then establish that the ensuing approximation is accurate relative to traditional linearization. Furthermore, we provide a rigorous example how this method can enable formal estimation of uncertainty in latent variables upstream of the network, within a target-tracking case study.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"188 ","pages":"Article 107376"},"PeriodicalIF":6.0,"publicationDate":"2025-03-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143704897","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Decomposition-based multi-scale transformer framework for time series anomaly detection

IF 6 1区计算机科学

Neural Networks Pub Date : 2025-03-20 DOI: 10.1016/j.neunet.2025.107399

Wenxin Zhang , Cuicui Luo

{"title":"Decomposition-based multi-scale transformer framework for time series anomaly detection","authors":"Wenxin Zhang , Cuicui Luo","doi":"10.1016/j.neunet.2025.107399","DOIUrl":"10.1016/j.neunet.2025.107399","url":null,"abstract":"<div><div>Time series anomaly detection is crucial for maintaining stable systems. Existing methods face two main challenges. First, it is difficult to directly model the dependencies of diverse and complex patterns within the sequences. Second, many methods that optimize parameters using mean squared error struggle with noise in the time series, leading to performance deterioration. To address these challenges, we propose a transformer-based framework built on decomposition (TransDe) for multivariate time series anomaly detection. The key idea is to combine the strengths of time series decomposition and transformers to effectively learn the complex patterns in normal time series data. A multi-scale patch-based transformer architecture is proposed to exploit the representative dependencies of each decomposed component of the time series. Furthermore, a contrastive learn paradigm based on patch operation is proposed, which leverages KL divergence to align the positive pairs, namely the pure representations of normal patterns between different patch-level views. A novel asynchronous loss function with a stop-gradient strategy is further introduced to enhance the performance of TransDe effectively. It can avoid time-consuming and labor-intensive computation costs in the optimization process. Extensive experiments on five public datasets are conducted and TransDe shows superiority compared with twelve baselines in terms of F1 score. Our code is available at <span><span>https://github.com/shaieesss/TransDe</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"187 ","pages":"Article 107399"},"PeriodicalIF":6.0,"publicationDate":"2025-03-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143679900","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0