Neural Networks最新文献

{"title":"Boosting semi-supervised federated learning by effectively exploiting server-side knowledge and client-side unconfident samples","authors":"Hongquan Liu ,&nbsp;Yuxi Mi ,&nbsp;Yateng Tang ,&nbsp;Jihong Guan ,&nbsp;Shuigeng Zhou","doi":"10.1016/j.neunet.2025.107440","DOIUrl":"10.1016/j.neunet.2025.107440","url":null,"abstract":"<div><div>Semi-supervised federated learning (SSFL) has emerged as a promising paradigm to reduce the need for fully labeled data in training federated learning (FL) models. This paper focuses on the label-at-server scenario, where clients’ data are entirely unlabeled and the server possesses only a limited amount of labeled data. In this setting, the non-independent and identically distributed (non-IID) local data and the incorrect pseudo-labels will possibly introduce bias into the model during local training. Prior works try to alleviate the bias by fine-tuning the global model with clean labeled data, ignoring explicitly leveraging server-side knowledge to guide local training. Additionally, existing methods typically discard samples with unconfident pseudo-labels, resulting in many samples being not used, consequently suboptimal performance and slow convergence. This paper introduces a novel method to enhance SSFL performance by effectively exploiting server-side clean knowledge and client-side unconfident samples. Specifically, we propose a representation alignment module that mitigates the influence of non-IID data by aligning local features with the <em>class proxies</em> of the server labeled data. Furthermore, we employ a shrink loss to reduce the risk associated with unreliable pseudo-labels, ensuring the exploitation of valuable information contained in the entire unlabeled dataset. Extensive experiments on five benchmark datasets under various settings demonstrate the effectiveness and generality of the proposed method, which not only outperforms existing methods but also reduces the communication cost required to achieve the target performance.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"188 ","pages":"Article 107440"},"PeriodicalIF":6.0,"publicationDate":"2025-04-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143821524","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"FSDM: An efficient video super-resolution method based on Frames-Shift Diffusion Model","authors":"Shijie Yang ,&nbsp;Chao Chen ,&nbsp;Jie Liu ,&nbsp;Jie Tang ,&nbsp;Gangshan Wu","doi":"10.1016/j.neunet.2025.107435","DOIUrl":"10.1016/j.neunet.2025.107435","url":null,"abstract":"<div><div>Video super-resolution is a fundamental task aimed at enhancing video quality through intricate modeling techniques. Recent advancements in diffusion models have significantly enhanced image super-resolution processing capabilities. However, their integration into video super-resolution workflows remains constrained due to the computational complexity of temporal fusion modules, demanding more computational resources compared to their image counterparts. To address this challenge, we propose a novel approach: a Frames-Shift Diffusion Model based on the image diffusion models. Compared to directly training diffusion-based video super-resolution models, redesigning the diffusion process of image models without introducing complex temporal modules requires minimal training consumption. We incorporate temporal information into the image super-resolution diffusion model by using optical flow and perform multi-frame fusion. This model adapts the diffusion process to smoothly transition from image super-resolution to video super-resolution diffusion without additional weight parameters. As a result, the Frames-Shift Diffusion Model efficiently processes videos frame by frame while maintaining computational efficiency and achieving superior performance. It enhances perceptual quality and achieves comparable performance to other state-of-the-art diffusion-based VSR methods in PSNR and SSIM. This approach optimizes video super-resolution by simplifying the integration of temporal data, thus addressing key challenges in the field.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"188 ","pages":"Article 107435"},"PeriodicalIF":6.0,"publicationDate":"2025-04-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143769264","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Localize-diffusion based dual-branch anomaly detection","authors":"Jielin Jiang ,&nbsp;Xiying Liu ,&nbsp;Peiyi Yan ,&nbsp;Shun Wei ,&nbsp;Yan Cui","doi":"10.1016/j.neunet.2025.107439","DOIUrl":"10.1016/j.neunet.2025.107439","url":null,"abstract":"<div><div>Due to the scarcity of real anomaly samples for use in anomaly detection studies, data augmentation methods are typically employed to generate pseudo anomaly samples to supplement the limited real samples. However, existing data augmentation methods often generate image patches with fixed shapes as anomalies in random regions. These anomalies are unrealistic and lack diversity, resulting in generated samples with limited practical value. To address this issue, we propose a dual-branch anomaly detection (DBA) technique based on Localize-Diffusion (LD) augmentation. LD can infer the approximate position and size of the object to be detected based on the samples’ color distribution: this can effectively avoid the problem of patch generation outside the target object’s location. LD subsequently incorporates hard augmentation and continuously propagates irregular patches to the surrounding area, which enriches the diversity of the generated samples. Based on the anomalies’ multi-scale characteristics, DBA adopts two branches for training and anomaly detection based on the generated pseudo anomaly samples: one focuses on identifying anomaly-specific features from learned anomalies, while the other discriminates between normal and anomaly samples based on residual features in the latent space. Finally, an adaptive scoring module is used to calculate a weighted average of the results of the two branches, achieving the goal of anomaly detection. Extensive experimental analyses reveal that DBA achieves excellent anomaly detection performance using only 14.2M parameters, notably achieving 99.6 detection AUC on the MVTec AD dataset.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"188 ","pages":"Article 107439"},"PeriodicalIF":6.0,"publicationDate":"2025-04-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143769163","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Enhancing multiview synergy: Robust learning by exploiting the wave loss function with consensus and complementarity principles","authors":"A. Quadir,&nbsp;Mushir Akhtar,&nbsp;M. Tanveer","doi":"10.1016/j.neunet.2025.107433","DOIUrl":"10.1016/j.neunet.2025.107433","url":null,"abstract":"<div><div>Multiview learning (MvL) is an advancing domain in machine learning, leveraging multiple data perspectives to enhance model performance through view-consistency and view-discrepancy. Despite numerous successful multiview-based support vector machine (SVM) models, existing frameworks predominantly focus on the consensus principle, often overlooking the complementarity principle. Furthermore, they exhibit limited robustness against noisy, error-prone, and view-inconsistent samples, prevalent in multiview datasets. To tackle the aforementioned limitations, this paper introduces Wave-MvSVM, a novel multiview support vector machine framework leveraging the wave loss (W-loss) function, specifically designed to harness both consensus and complementarity principles. Unlike traditional approaches that often overlook the complementary information among different views, the proposed Wave-MvSVM ensures a more comprehensive and resilient learning process by integrating both principles effectively. The W-loss function, characterized by its smoothness, asymmetry, and bounded nature, is particularly effective in mitigating the adverse effects of noisy and outlier data, thereby enhancing model stability. Theoretically, the W-loss function also exhibits a crucial classification-calibrated property, further boosting its effectiveness. The proposed Wave-MvSVM employs a between-view co-regularization term to enforce view consistency and utilizes an adaptive combination weight strategy to maximize the discriminative power of each view, thus fully exploiting both consensus and complementarity principles. The optimization problem is efficiently solved using a combination of gradient descent (GD) and the alternating direction method of multipliers (ADMM), ensuring reliable convergence to optimal solutions. The generalization abilities of the proposed Wave-MvSVM model is theoretically supported through analyses based on Rademacher complexity. Extensive empirical evaluations across diverse datasets demonstrate the superior performance of Wave-MvSVM in comparison to existing benchmark models, highlighting its potential as a robust and efficient solution for MvL challenges. Furthermore, we implemented the proposed Wave-MvSVM model on Schizophrenia dataset, showcasing the model’s efficacy in real-world applications. The source code of the proposed Wave-MvSVM model is available at <span><span>https://github.com/mtanveer1/Wave-MvSVM</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"188 ","pages":"Article 107433"},"PeriodicalIF":6.0,"publicationDate":"2025-04-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143799581","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"What is the doggest dog? Examination of typicality perception in ImageNet-trained networks","authors":"Katarzyna Filus,&nbsp;Joanna Domańska","doi":"10.1016/j.neunet.2025.107425","DOIUrl":"10.1016/j.neunet.2025.107425","url":null,"abstract":"<div><div>Due to the emergence of numerous model architectures in recent years, researchers finally have access to models that are diverse enough to properly study them from the perspective of cognitive psychology theories, e.g. Prototype Theory. The theory assumes that the degree of membership in a basic-level category is graded. As a result, some concepts are perceived as more central (typical) than others. The most typical category is called a prototype. It can be perceived as the clearest example of a category, reflecting the redundancy structure of the category as a whole. Its inverse is called an anti-prototype. Reasonable perception of prototypes and anti-prototypes is important for accurate projection of the world structure onto the class space and more human-like world perception beyond simple memorization. That is why it is beneficial to study deep models from the perspective of prototype theory. To enable it, we propose 3 methods that return the prototypes and anti-prototypes perceived by deep networks for a specific basic-level category. Additionally, one of our methods allows to visualize the centrality of objects. The results on a wide range of 42 networks trained on ImageNet (Convolutional Networks, Vision Transformers, ConvNeXts and hybrid models) reveal that the networks share the typicality perception to a large extent and that this perception does not lie so far from the human one. We release the dataset with per-network prototypes and anti-prototypes resulting from our work to enable further research on this topic.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"188 ","pages":"Article 107425"},"PeriodicalIF":6.0,"publicationDate":"2025-04-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143817122","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Source-free time series domain adaptation with wavelet-based multi-scale temporal imputation","authors":"Yingyi Zhong,&nbsp;Wen’an Zhou,&nbsp;Liwen Tao","doi":"10.1016/j.neunet.2025.107428","DOIUrl":"10.1016/j.neunet.2025.107428","url":null,"abstract":"<div><div>Recent works on source-free domain adaptation (SFDA) for time series reveal the effectiveness of learning domain-invariant temporal dynamics on improving the cross-domain performance of the model. However, existing SFDA methods for time series mainly focus on modeling the original sequence, lacking the utilization of the multi-scale properties of time series. This may result in insufficient extraction of domain-invariant temporal patterns. Furthermore, previous multi-scale analysis methods typically ignore important frequency domain information during multi-scale division, leading to the limited ability for multi-scale time series modeling. To this end, we propose LEMON, a novel SFDA method for time series with wavelet-based multi-scale temporal imputation. It utilizes the discrete wavelet transform to decompose a time series into multiple scales, each with a distinct time–frequency resolution and specific frequency range, enabling full-spectrum utilization. To effectively transfer multi-scale temporal dynamics from the source domain to the target domain, we introduce a multi-scale temporal imputation module which assigns a deep neural network to perform the temporal imputation task on the sequence at each scale, learning scale-specific domain-invariant information. We further design an energy-based multi-scale weighting strategy, which adaptively integrates information from multiple scales based on the frequency distribution of the input data to improve the transfer performance of the model. Extensive experiments on three real-world time series datasets demonstrate that LEMON significantly outperforms the state-of-the-art methods, achieving an average improvement of 4.45% in accuracy and 6.29% in MF1-score.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"188 ","pages":"Article 107428"},"PeriodicalIF":6.0,"publicationDate":"2025-04-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143759541","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Multi-level feature fusion networks for smoke recognition in remote sensing imagery.","authors":"Yupeng Wang, Yongli Wang, Zaki Ahmad Khan, Anqi Huang, Jianghui Sang","doi":"10.1016/j.neunet.2024.107112","DOIUrl":"10.1016/j.neunet.2024.107112","url":null,"abstract":"<p><p>Smoke is a critical indicator of forest fires, often detectable before flames ignite. Accurate smoke identification in remote sensing images is vital for effective forest fire monitoring within Internet of Things (IoT) systems. However, existing detection methods frequently falter in complex real-world scenarios, where variable smoke shapes and sizes, intricate backgrounds, and smoke-like phenomena (e.g., clouds and haze) lead to missed detections and false alarms. To address these challenges, we propose the Multi-level Feature Fusion Network (MFFNet), a novel framework grounded in contrastive learning. MFFNet begins by extracting multi-scale features from remote sensing images using a pre-trained ConvNeXt model, capturing information across different levels of granularity to accommodate variations in smoke appearance. The Attention Feature Enhancement Module further refines these multi-scale features, enhancing fine-grained, discriminative attributes relevant to smoke detection. Subsequently, the Bilinear Feature Fusion Module combines these enriched features, effectively reducing background interference and improving the model's ability to distinguish smoke from visually similar phenomena. Finally, contrastive feature learning is employed to improve robustness against intra-class variations by focusing on unique regions within the smoke patterns. Evaluated on the benchmark dataset USTC_SmokeRS, MFFNet achieves an accuracy of 98.87%. Additionally, our model demonstrates a detection rate of 94.54% on the extended E_SmokeRS dataset, with a low false alarm rate of 3.30%. These results highlight the effectiveness of MFFNet in recognizing smoke in remote sensing images, surpassing existing methodologies. The code is accessible at https://github.com/WangYuPeng1/MFFNet.</p>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"184 ","pages":"107112"},"PeriodicalIF":6.0,"publicationDate":"2025-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142967303","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Towards a better evaluation of out-of-domain generalization","authors":"Duhun Hwang ,&nbsp;Suhyun Kang ,&nbsp;Moonjung Eo ,&nbsp;Jimyeong Kim ,&nbsp;Wonjong Rhee","doi":"10.1016/j.neunet.2025.107434","DOIUrl":"10.1016/j.neunet.2025.107434","url":null,"abstract":"<div><div>The objective of Domain Generalization (DG) is to devise algorithms capable of achieving high performance on previously unseen test distributions. In the pursuit of this objective, average measure has been employed as the prevalent measure for comparing algorithms in the existing DG studies. Despite its significance, a comprehensive exploration of the average measure has been lacking and its suitability in approximating the true domain generalization performance has been questionable. In this study, we carefully investigate the limitations inherent in the average measure and propose worst+gap measure as a robust alternative. We establish theoretical grounds of the proposed measure by deriving two theorems starting from two different assumptions. Despite the independence in the two assumptions, we will show that both theorems lead to a common insight. We conduct extensive experimental investigations to compare the proposed worst+gap measure with the conventional average measure. Given the indispensable need to access the true DG performance for studying measures, we modify five existing datasets to come up with SR-CMNIST, C-Cats&amp;Dogs, L-CIFAR10, PACS-corrupted, and VLCS-corrupted datasets. The experiment results unveil an inferior performance of the average measure in approximating the true DG performance and confirm the robustness of the theoretically supported worst+gap measure.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"188 ","pages":"Article 107434"},"PeriodicalIF":6.0,"publicationDate":"2025-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143783215","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"ICH-PRNet: a cross-modal intracerebral haemorrhage prognostic prediction method using joint-attention interaction mechanism.","authors":"Xinlei Yu, Ahmed Elazab, Ruiquan Ge, Jichao Zhu, Lingyan Zhang, Gangyong Jia, Qing Wu, Xiang Wan, Lihua Li, Changmiao Wang","doi":"10.1016/j.neunet.2024.107096","DOIUrl":"10.1016/j.neunet.2024.107096","url":null,"abstract":"<p><p>Accurately predicting intracerebral hemorrhage (ICH) prognosis is a critical and indispensable step in the clinical management of patients post-ICH. Recently, integrating artificial intelligence, particularly deep learning, has significantly enhanced prediction accuracy and alleviated neurosurgeons from the burden of manual prognosis assessment. However, uni-modal methods have shown suboptimal performance due to the intricate pathophysiology of the ICH. On the other hand, existing cross-modal approaches that incorporate tabular data have often failed to effectively extract complementary information and cross-modal features between modalities, thereby limiting their prognostic capabilities. This study introduces a novel cross-modal network, ICH-PRNet, designed to predict ICH prognosis outcomes. Specifically, we propose a joint-attention interaction encoder that effectively integrates computed tomography images and clinical texts within a unified representational space. Additionally, we define a multi-loss function comprising three components to comprehensively optimize cross-modal fusion capabilities. To balance the training process, we employ a self-adaptive dynamic prioritization algorithm that adjusts the weights of each component, accordingly. Our model, through these innovative designs, establishes robust semantic connections between modalities and uncovers rich, complementary cross-modal information, thereby achieving superior prediction results. Extensive experimental results and comparisons with state-of-the-art methods on both in-house and publicly available datasets unequivocally demonstrate the superiority and efficacy of the proposed method. Our code is at https://github.com/YU-deep/ICH-PRNet.git.</p>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"184 ","pages":"107096"},"PeriodicalIF":6.0,"publicationDate":"2025-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142972996","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Neighborhood relation-based knowledge distillation for image classification","authors":"Jianping Gou ,&nbsp;Xiaomeng Xin ,&nbsp;Baosheng Yu ,&nbsp;Heping Song ,&nbsp;Weiyong Zhang ,&nbsp;Shaohua Wan","doi":"10.1016/j.neunet.2025.107429","DOIUrl":"10.1016/j.neunet.2025.107429","url":null,"abstract":"<div><div>As an efficient model compression method, recent knowledge distillation methods primarily transfer the knowledge from a large teacher model to a small student model by minimizing the differences between the predictions from teacher and student. However, the relationship between different samples has not been well-investigated, since recent relational distillation methods mainly construct the knowledge from all randomly selected samples, e.g., the similarity matrix of mini-batch samples. In this paper, we propose <strong>N</strong>eighborhood <strong>R</strong>elation-Based <strong>K</strong>nowledge <strong>D</strong>istillation (NRKD) to consider the local structure as the novel relational knowledge for better knowledge transfer. Specifically, we first find a subset of samples with their <span><math><mi>K</mi></math></span>-nearest neighbors according to the similarity matrix of mini-batch samples and then build the neighborhood relationship knowledge for knowledge distillation, where the characterized relational knowledge can be transferred by both intermediate feature maps and output logits. We perform extensive experiments on several popular image classification datasets for knowledge distillation, including CIFAR10, CIFAR100, Tiny ImageNet, and ImageNet. Experimental results demonstrate that the proposed NRKD yields competitive results, compared to the state-of-the art distillation methods. Our codes are available at: <span><span>https://github.com/xinxiaoxiaomeng/NRKD.git</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":49763,"journal":{"name":"Neural Networks","volume":"188 ","pages":"Article 107429"},"PeriodicalIF":6.0,"publicationDate":"2025-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143759539","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}