Neural Processing Letters最新文献_第4页

A Single Image High-Perception Super-Resolution Reconstruction Method Based on Multi-layer Feature Fusion Model with Adaptive Compression and Parameter Tuning 基于自适应压缩和参数调整的多层特征融合模型的单幅图像高感知超分辨率重建方法

IF 3.1 4区计算机科学

Neural Processing Letters Pub Date : 2024-06-19 DOI: 10.1007/s11063-024-11660-7

Rui Zhang, Wenyu Ren, Lihu Pan, Xiaolu Bai, Ji Li

{"title":"A Single Image High-Perception Super-Resolution Reconstruction Method Based on Multi-layer Feature Fusion Model with Adaptive Compression and Parameter Tuning","authors":"Rui Zhang, Wenyu Ren, Lihu Pan, Xiaolu Bai, Ji Li","doi":"10.1007/s11063-024-11660-7","DOIUrl":"https://doi.org/10.1007/s11063-024-11660-7","url":null,"abstract":"We propose a simple image high-perception super-resolution reconstruction method based on multi-layer feature fusion model with adaptive compression and parameter tuning. The aim is to further balance the high and low-frequency information of an image, enrich the detailed texture to improve perceptual quality, and improve the adaptive optimization and generalization of the model in the process of super-resolution reconstruction. First, an effective multi-layer fusion super-resolution (MFSR) basic model is constructed by the design of edge enhancement, refine layering, enhanced super-resolution generative adversarial network and other sub-models, and effective multi-layer fusion. This further enriches the image representation of features of different scales and depths and improves the feature representation of high and low-frequency information in a balanced way. Next, a total loss function of the generator is constructed with adaptive parameter tuning performance. The overall adaptability of the model is improved through adaptive weight distribution and fusion of content loss, perceptual loss, and adversarial loss, and improving the error while reducing the edge enhancement model. Finally, a fitness function with the evaluation perceptual function as the optimization strategy is constructed, and the model compression and adaptive tuning of MFSR are carried out based on the multi-mechanism fusion strategy. Consequently, the construction of the adaptive MFSR model is realized. Adaptive MFSR can maintain high peak signal to noise ratio and structural similarity on the test sets Set5, Set14, and BSD100, and achieve high-quality reconstructed images with low learned perceptual image patch similarity and perceptual index, while having good generalization capabilities.","PeriodicalId":51144,"journal":{"name":"Neural Processing Letters","volume":"144 1","pages":""},"PeriodicalIF":3.1,"publicationDate":"2024-06-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141505309","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Multi-objective Evolutionary Neural Architecture Search for Recurrent Neural Networks 递归神经网络的多目标进化神经架构搜索

IF 3.1 4区计算机科学

Neural Processing Letters Pub Date : 2024-06-18 DOI: 10.1007/s11063-024-11659-0

Reinhard Booysen, Anna Sergeevna Bosman

{"title":"Multi-objective Evolutionary Neural Architecture Search for Recurrent Neural Networks","authors":"Reinhard Booysen, Anna Sergeevna Bosman","doi":"10.1007/s11063-024-11659-0","DOIUrl":"https://doi.org/10.1007/s11063-024-11659-0","url":null,"abstract":"Artificial neural network (NN) architecture design is a nontrivial and time-consuming task that often requires a high level of human expertise. Neural architecture search (NAS) serves to automate the design of NN architectures and has proven to be successful in automatically finding NN architectures that outperform those manually designed by human experts. NN architecture performance can be quantified based on multiple objectives, which include model accuracy and some NN architecture complexity objectives, among others. The majority of modern NAS methods that consider multiple objectives for NN architecture performance evaluation are concerned with automated feed forward NN architecture design, which leaves multi-objective automated recurrent neural network (RNN) architecture design unexplored. RNNs are important for modeling sequential datasets, and prominent within the natural language processing domain. It is often the case in real world implementations of machine learning and NNs that a reasonable trade-off is accepted for marginally reduced model accuracy in favour of lower computational resources demanded by the model. This paper proposes a multi-objective evolutionary algorithm-based RNN architecture search method. The proposed method relies on approximate network morphisms for RNN architecture complexity optimisation during evolution. The results show that the proposed method is capable of finding novel RNN architectures with comparable performance to state-of-the-art manually designed RNN architectures, but with reduced computational demand.\u0000","PeriodicalId":51144,"journal":{"name":"Neural Processing Letters","volume":"202 1","pages":""},"PeriodicalIF":3.1,"publicationDate":"2024-06-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141520319","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A Random Focusing Method with Jensen–Shannon Divergence for Improving Deep Neural Network Performance Ensuring Architecture Consistency 用詹森-香农发散法提高深度神经网络性能的随机聚焦法确保架构一致性

IF 3.1 4区计算机科学

Neural Processing Letters Pub Date : 2024-06-17 DOI: 10.1007/s11063-024-11668-z

Wonjik Kim

{"title":"A Random Focusing Method with Jensen–Shannon Divergence for Improving Deep Neural Network Performance Ensuring Architecture Consistency","authors":"Wonjik Kim","doi":"10.1007/s11063-024-11668-z","DOIUrl":"https://doi.org/10.1007/s11063-024-11668-z","url":null,"abstract":"Multiple hidden layers in deep neural networks perform non-linear transformations, enabling the extraction of meaningful features and the identification of relationships between input and output data. However, the gap between the training and real-world data can result in network overfitting, prompting the exploration of various preventive methods. The regularization technique called ’dropout’ is widely used for deep learning models to improve the training of robust and generalized features. During the training phase with dropout, neurons in a particular layer are randomly selected to be ignored for each input. This random exclusion of neurons encourages the network to depend on different subsets of neurons at different times, fostering robustness and reducing sensitivity to specific neurons. This study introduces a novel approach called random focusing, departing from complete neuron exclusion in dropout. The proposed random focusing selectively highlights random neurons during training, aiming for a smoother transition between training and inference phases while keeping network architecture consistent. This study also incorporates Jensen–Shannon Divergence to enhance the stability and efficacy of the random focusing method. Experimental validation across tasks like image classification and semantic segmentation demonstrates the adaptability of the proposed methods across different network architectures, including convolutional neural networks and transformers.\u0000","PeriodicalId":51144,"journal":{"name":"Neural Processing Letters","volume":"1 1","pages":""},"PeriodicalIF":3.1,"publicationDate":"2024-06-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141505311","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Add-Vit: CNN-Transformer Hybrid Architecture for Small Data Paradigm Processing Add-Vit：用于小型数据范式处理的 CNN-变压器混合架构

IF 3.1 4区计算机科学

Neural Processing Letters Pub Date : 2024-06-07 DOI: 10.1007/s11063-024-11643-8

Jinhui Chen, Peng Wu, Xiaoming Zhang, Renjie Xu, Jia Liang

引用次数: 0

Knowledge Distillation Based on Narrow-Deep Networks 基于窄深网络的知识提炼

IF 3.1 4区计算机科学

Neural Processing Letters Pub Date : 2024-06-06 DOI: 10.1007/s11063-024-11646-5

Yan Zhou, Zhiqiang Wang, Jianxun Li

引用次数: 0

Nested Entity Recognition Method Based on Multidimensional Features and Fuzzy Localization 基于多维特征和模糊定位的嵌套实体识别方法

IF 3.1 4区计算机科学

Neural Processing Letters Pub Date : 2024-06-04 DOI: 10.1007/s11063-024-11657-2

Hua Zhao, Xueyang Bai, Qingtian Zeng, Heng Zhou, Xuemei Bai

{"title":"Nested Entity Recognition Method Based on Multidimensional Features and Fuzzy Localization","authors":"Hua Zhao, Xueyang Bai, Qingtian Zeng, Heng Zhou, Xuemei Bai","doi":"10.1007/s11063-024-11657-2","DOIUrl":"https://doi.org/10.1007/s11063-024-11657-2","url":null,"abstract":"Nested named entity recognition (NNER) aims to identify potentially overlapping named entities. Sequence labeling method and span-based method are two commonly used methods in nested named entity recognition. However, the linear structure of sequence labeling method results in relatively poor performance, and span-based method requires traversing all spans, which brings very high time complexity. All of them fail to effectively leverage the positional dependencies between internal and external entities. In order to improve these issues, this paper proposed a nested entity recognition method based on Multidimensional Features and Fuzzy Localization (MFFL). Firstly, this method adopted the shared encoding that fused three features of characters, words, and parts of speech to obtain a multidimensional feature vector representation of the text and obtained rich semantic information in the text. Secondly, we proposed to use the fuzzy localization to assist the model in pinpointing the potential locations of entities. Finally, in the entity classification, it used a window to expand the sub-sequence and enumerate possible candidate entities and predicted the classification labels of these candidate entities. In order to alleviate the problem of error propagation and effectively learn the correlation between fuzzy localization and classification labels, we adopted multi-task learning strategy. This paper conducted several experiments on two public datasets. The experimental results showed that the proposed method achieves ideal results in both nested entity recognition and non-nested entity recognition tasks, and significantly reduced the time complexity of nested entity recognition.","PeriodicalId":51144,"journal":{"name":"Neural Processing Letters","volume":"121 1","pages":""},"PeriodicalIF":3.1,"publicationDate":"2024-06-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141253147","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Capsule Network Based on Double-layer Attention Mechanism and Multi-scale Feature Extraction for Remaining Life Prediction 基于双层注意机制和多尺度特征提取的胶囊网络用于剩余寿命预测

IF 3.1 4区计算机科学

Neural Processing Letters Pub Date : 2024-06-03 DOI: 10.1007/s11063-024-11651-8

Zhiwu Shang, Zehua Feng, Wanxiang Li, Zhihua Wu, Hongchuan Cheng

{"title":"Capsule Network Based on Double-layer Attention Mechanism and Multi-scale Feature Extraction for Remaining Life Prediction","authors":"Zhiwu Shang, Zehua Feng, Wanxiang Li, Zhihua Wu, Hongchuan Cheng","doi":"10.1007/s11063-024-11651-8","DOIUrl":"https://doi.org/10.1007/s11063-024-11651-8","url":null,"abstract":"The era of big data provides a platform for high-precision RUL prediction, but the existing RUL prediction methods, which effectively extract key degradation information, remain a challenge. Existing methods ignore the influence of sensor and degradation moment variability, and instead assign weights to them equally, which affects the final prediction accuracy. In addition, convolutional networks lose key information due to downsampling operations and also suffer from the drawback of insufficient feature extraction capability. To address these issues, the two-layer attention mechanism and the Inception module are embedded in the capsule structure (mai-capsule model) for lifetime prediction. The first layer of the channel attention mechanism (CAM) evaluates the influence of various sensor information on the forecast; the second layer adds a time-step attention (TSAM) mechanism to the LSTM network to weigh the contribution of different moments of the engine's whole life cycle to the prediction, while weakening the influence of environmental noise on the prediction. The Inception module is introduced to perform multi-scale feature extraction on the weighted data to capture the degradation information to the maximum extent. Lastly, we are inspired to employ the capsule network to capture important position information of high and low-dimensional features, given its capacity to facilitate a more effective rendition of the overall features of the time-series data. The efficacy of the suggested model is assessed against other approaches and verified using the publicly accessible C-MPASS dataset. The end findings demonstrate the excellent prediction precision of the suggested approach.","PeriodicalId":51144,"journal":{"name":"Neural Processing Letters","volume":"98 1","pages":""},"PeriodicalIF":3.1,"publicationDate":"2024-06-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141253019","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Temporal Diversity-Aware Micro-Video Recommendation with Long- and Short-Term Interests Modeling 利用长短期兴趣建模的时态多样性感知微视频推荐

IF 3.1 4区计算机科学

Neural Processing Letters Pub Date : 2024-06-03 DOI: 10.1007/s11063-024-11652-7

Pan Gu, Haiyang Hu, Dongjing Wang, Dongjin Yu, Guandong Xu

{"title":"Temporal Diversity-Aware Micro-Video Recommendation with Long- and Short-Term Interests Modeling","authors":"Pan Gu, Haiyang Hu, Dongjing Wang, Dongjin Yu, Guandong Xu","doi":"10.1007/s11063-024-11652-7","DOIUrl":"https://doi.org/10.1007/s11063-024-11652-7","url":null,"abstract":"Recommender systems have become indispensable for addressing information overload for micro-video services. They are used to characterize users’ preferences from their historical interactions and recommend micro-videos accordingly. Existing works largely leverage the multi-modal contents of micro-videos to enhance recommendation performance. However, limited efforts have been made to understand users’ complex behavior patterns, including their long- and short-term interests, as well as their temporal diversity preferences. In micro-video recommendation scenarios, users tend to have both stable long-term interests and dynamic short-term interests, and may feel tired after incessantly receiving numerous similar recommendations. In this paper, we propose a Temporal Diversity-aware micro-video recommender (TD-VideoRec) for user behavior modeling, simultaneously capturing users’ long- and short-term preferences. Specifically, we first adopt a user-centric attention mechanism to cope with long-term interests. Then, we utilize an attention network on top of a long-short term memory network to obtain users’ short-term interests. Finally, a temporal diversity coefficient is introduced to characterize the temporal diversity preferences of users’ click behaviors. The value of the coefficient depends on the distinction between users’ long- and short-term interests extracted by vector orthogonal projection. Extensive experiments on two real-world datasets demonstrate that TD-VideoRec outperforms state-of-the-art methods.","PeriodicalId":51144,"journal":{"name":"Neural Processing Letters","volume":"3 1","pages":""},"PeriodicalIF":3.1,"publicationDate":"2024-06-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141252848","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

SpanEffiDet: Span-Scale and Span-Path Feature Fusion for Object Detection SpanEffiDet：用于物体检测的跨尺度和跨路径特征融合

IF 3.1 4区计算机科学

Neural Processing Letters Pub Date : 2024-06-02 DOI: 10.1007/s11063-024-11653-6

Qunpo Liu, Yi Zhao, Ruxin Gao, Xuhui Bu, Naohiko Hanajima

{"title":"SpanEffiDet: Span-Scale and Span-Path Feature Fusion for Object Detection","authors":"Qunpo Liu, Yi Zhao, Ruxin Gao, Xuhui Bu, Naohiko Hanajima","doi":"10.1007/s11063-024-11653-6","DOIUrl":"https://doi.org/10.1007/s11063-024-11653-6","url":null,"abstract":"Lower versions of EfficientDet (such as D0, D1) have smaller network structures and parameter sizes, but lower detection accuracy. Higher versions exhibit higher accuracy, but the increase in network complexity poses challenges for real-time processing and hardware requirements. To meet the higher accuracy requirements under limited computational resources, this paper introduces SpanEffiDet based on the channel adaptive frequency filter (CAFF) and the Span-Path Bidirectional Feature Pyramid structure. Firstly, the CAFF module proposed in this paper realizes the frequency domain transformation of channel information through Fourier transform and effectively extracts the key features through semantic adaptive frequency filtering, thus, eliminating channel redundant information of EfficientNet. Simultaneously, the module has the ability to compute the weights across the channels and at fine granularity, and capture the detailed information of element features. Secondly, a two-way characteristic pyramid network multi-level cross-BIFPN, which can achieve multi-layer and multi-nodes, is proposed to build cross-level information transmission to incorporate both semantic and positional information of the target. This design enables the network to more effectively detect objects with significant size differences in complex environments. Finally, by introducing generalized focal Loss V2, reliable localization quality estimation scores are predicted from the distribution statistics of bounding boxes, thereby improving localization accuracy. The experimental results indicate that on the MS COCO dataset, SpanEffiDet-D0 achieved an AP improvement of 3.3% compared to the original EfficientDet series algorithms. Similarly, on the PASCAL VOC2007 and 2012 datasets, the mAP of SpanEffiDet-D0 is respectively 1.66 and 2.65% higher than that of EfficientDet-D0.","PeriodicalId":51144,"journal":{"name":"Neural Processing Letters","volume":"98 1","pages":""},"PeriodicalIF":3.1,"publicationDate":"2024-06-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141253265","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

GPINN with Neural Tangent Kernel Technique for Nonlinear Two Point Boundary Value Problems 针对非线性两点边值问题的 GPINN 与神经切线核技术

IF 3.1 4区计算机科学

Neural Processing Letters Pub Date : 2024-05-31 DOI: 10.1007/s11063-024-11644-7

Navnit Jha, Ekansh Mallik

{"title":"GPINN with Neural Tangent Kernel Technique for Nonlinear Two Point Boundary Value Problems","authors":"Navnit Jha, Ekansh Mallik","doi":"10.1007/s11063-024-11644-7","DOIUrl":"https://doi.org/10.1007/s11063-024-11644-7","url":null,"abstract":"Neural networks as differential equation solvers are a good choice of numerical technique because of their fast solutions and their nature in tackling some classical problems which traditional numerical solvers faced. In this article, we look at the famous gradient descent optimization technique, which trains the network by updating parameters which minimizes the loss function. We look at the theoretical part of gradient descent to understand why the network works great for some terms of the loss function and not so much for other terms. The loss function considered here is built in such a way that it incorporates the differential equation as well as the derivative of the differential equation. The fully connected feed-forward network is designed in such a way that, without training at boundary points, it automatically satisfies the boundary conditions. The neural tangent kernel for gradient enhanced physics informed neural networks is examined in this work, and we demonstrate how it may be used to generate a closed-form expression for the kernel function. We also provide numerical experiments demonstrating the effectiveness of the new approach for several two point boundary value problems. Our results suggest that the neural tangent kernel based approach can significantly improve the computational accuracy of the gradient enhanced physics informed neural network while reducing the computational cost of training these models.","PeriodicalId":51144,"journal":{"name":"Neural Processing Letters","volume":"15 1","pages":""},"PeriodicalIF":3.1,"publicationDate":"2024-05-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141191931","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":4,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0