Applied Soft Computing最新文献_第5页

Bi-Branching Feature Interaction Representation Learning for Multivariate Time Series 多变量时间序列的双分支特征交互表征学习

IF 7.2 1区计算机科学

Applied Soft Computing Pub Date : 2024-10-30 DOI: 10.1016/j.asoc.2024.112383

Wenyan Wang , Enguang Zuo , Ruiting Wang , Jie Zhong , Chen Chen , Cheng Chen , Xiaoyi Lv

{"title":"Bi-Branching Feature Interaction Representation Learning for Multivariate Time Series","authors":"Wenyan Wang , Enguang Zuo , Ruiting Wang , Jie Zhong , Chen Chen , Cheng Chen , Xiaoyi Lv","doi":"10.1016/j.asoc.2024.112383","DOIUrl":"10.1016/j.asoc.2024.112383","url":null,"abstract":"<div><div>Representational learning of time series plays a crucial role in various fields. However, existing time-series models do not perform well in representation learning. These models usually focus only on the relationship between variables at the same timestamp or only consider the change of individual variables in time, while failing to effectively integrate the two, which limits their ability to capture complex time dependencies and multivariate interactions. We propose a <strong>Bi</strong>-Branching <strong>F</strong>eature <strong>I</strong>nteraction Representation Learning for Multivariate Time Series (Bi-FI) to address these issues. Specifically, we elaborated a frequency domain analysis branch to address the complex associations between variables that are difficult to visualize in the time domain. In addition, to eliminate the time lag effect, another branch employs the mechanism of variable tokenization for attention to learning intra-variable representations. Ultimately, we interactively fuse the features learned from the two branches to obtain a more comprehensive time series representation. Bi-FI performs well in three time series analysis tasks: long sequence prediction, classification, and anomaly detection. Our code and dataset will be available at <span><span>https://github.com/wwy8/Bi_FI</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":50737,"journal":{"name":"Applied Soft Computing","volume":"167 ","pages":"Article 112383"},"PeriodicalIF":7.2,"publicationDate":"2024-10-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142656805","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Robust Chinese Clinical Named Entity Recognition with information bottleneck and adversarial training 利用信息瓶颈和对抗训练实现鲁棒的中文临床命名实体识别

IF 7.2 1区计算机科学

Applied Soft Computing Pub Date : 2024-10-30 DOI: 10.1016/j.asoc.2024.112409

Yunfei He , Zhiqiang Zhang , Jinlong Shen , Yuling Li , Yiwen Zhang , Weiping Ding , Fei Yang

{"title":"Robust Chinese Clinical Named Entity Recognition with information bottleneck and adversarial training","authors":"Yunfei He , Zhiqiang Zhang , Jinlong Shen , Yuling Li , Yiwen Zhang , Weiping Ding , Fei Yang","doi":"10.1016/j.asoc.2024.112409","DOIUrl":"10.1016/j.asoc.2024.112409","url":null,"abstract":"<div><div>Chinese Clinical Named Entity Recognition (CCNER) aims to extract entities with specific medical significance from Chinese clinical texts, which is an important part of medical data mining. Some existing CCNER models may assume perfect text data and design complex models to improve their accuracy. However, due to the complexity of Chinese clinical entity semantics and the professionalism of annotation, Chinese clinical texts are prone to contain irregular misrepresentations and sparse entity labeling. That would lead to noisy or incomplete text features extracted by CCNER, seriously threatening the robustness of recognition in real-world scenarios. To address these problems, we propose the Robust Chinese Clinical Named Entity Recognition model (RCCNER). RCCNER comprises three essential components: multifaceted text representation, robust feature extraction, and robust model training. For multifaceted text representation, the model enhances consistency and collaboration between feature representations by integrating word embedding, radical embedding, and dictionary embedding to help withstand textual noise. Then, guided by the information bottleneck and the Hilbert–Schmidt independence criterion, robust feature extraction compresses the dependency between text representation and extracted features, while enhancing the dependency between extracted features and labels, which consequently provides reliable text features for robust recognition. The robust model training aspect leverages adversarial training to diminish RCCNER’s sensitivity to noise disturbances and sparse entity labeling, thereby reinforcing its robustness in entity recognition. RCCNER collaboratively enhances the noise immunity through text representation, text feature extraction and model training. Several experiments on two popular public datasets validate the effectiveness and robustness of RCCNER.</div></div>","PeriodicalId":50737,"journal":{"name":"Applied Soft Computing","volume":"167 ","pages":"Article 112409"},"PeriodicalIF":7.2,"publicationDate":"2024-10-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142578863","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Clustering based fuzzy classification with a noise cluster in detecting fraud in insurance 基于聚类的模糊分类与噪声聚类在检测保险欺诈中的应用

IF 7.2 1区计算机科学

Applied Soft Computing Pub Date : 2024-10-30 DOI: 10.1016/j.asoc.2024.112430

Oguz Koc , Furkan Baser , A. Sevtap Selcuk-Kestel

{"title":"Clustering based fuzzy classification with a noise cluster in detecting fraud in insurance","authors":"Oguz Koc , Furkan Baser , A. Sevtap Selcuk-Kestel","doi":"10.1016/j.asoc.2024.112430","DOIUrl":"10.1016/j.asoc.2024.112430","url":null,"abstract":"<div><div>Fraud detection is one of the main issues in reducing the unsystematic risks in insurance business as its costs might reach to catastrophic amounts leading to higher loadings on reserves and premiums. Due to its cause of nature in diversity, fraud detection may require a wide range of factors and variables to be considered. To make logical relations between many factors and reveal their differences, estimate odds (or probabilities), and predict the fraud risk, scoring systems become an important aid. In this paper, we introduce a clustering-based fuzzy classification with a noise cluster (CBFCN) to identify the true state of a fraud. The approach proposed in this paper is based on fuzzy k-means clustering having a noise cluster (FKMN) and is a novel method for identifying outliers by achieving robust clustering. We integrate fuzzy theory to boost the prediction ability of machine learning (ML) approaches for a proper determination of the contributing features. The two critical features of the CBFCN method which are the membership values obtained from the FKMN clustering algorithm are implemented to capture the behavior of an existing structure better and detect the noise (extremes) in the dataset. Extensive analyses are made on two real datasets exposing different characteristics in their variables to demonstrate how CBFCN performs in detecting the fraud compared to the conventional approaches. Additionally, employing fuzzy approach to improve the ML performance is elaborated through the inclusion of noise clusters. The findings indicate that the suggested CBFCN models produce promising classification results in fraud detection in insurance claims occurrences.</div></div>","PeriodicalId":50737,"journal":{"name":"Applied Soft Computing","volume":"167 ","pages":"Article 112430"},"PeriodicalIF":7.2,"publicationDate":"2024-10-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142578868","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Clustering by detecting skeletal structure and identifying density fluctuation 通过检测骨骼结构和识别密度波动进行聚类

IF 7.2 1区计算机科学

Applied Soft Computing Pub Date : 2024-10-30 DOI: 10.1016/j.asoc.2024.112432

Wenjie Guo, Wei Chen, Xinggao Liu

{"title":"Clustering by detecting skeletal structure and identifying density fluctuation","authors":"Wenjie Guo, Wei Chen, Xinggao Liu","doi":"10.1016/j.asoc.2024.112432","DOIUrl":"10.1016/j.asoc.2024.112432","url":null,"abstract":"<div><div>Clustering is one of the most important techniques for unsupervised learning, it tries to divide points into different clusters without any priori knowledge of data. Therefore, several criterions for clustering algorithm are as follows: 1. Handling clusters with arbitrary shape and various density; 2. Finding cluster centers automatically; 3. Low parameter sensitivity and computational complexity. In this context, a novel algorithm namely clustering by detecting skeletal structure and identifying density fluctuation (CSSDF) was presented. In CSSDF, an efficient strategy based on density and local information of neighborhood is firstly proposed to detect the skeletal structure, which can collect the local information and identify the rough distribution of data. With the identified distribution information, a method takes expanded neighborhood and density fluctuation into consideration is proposed to further collect global information of data, which can assign all skeleton points and find cluster centers. To sum up, CSSDF can not only discover the underlying structure of data regardless of its’ distribution, but also ensure the correct assignment of all skeleton points and thus lead to a satisfying clustering performance. In addition, the computational complexity of the proposed approach is <span><math><mrow><mi>O</mi><mo>(</mo><mi>nlogn</mi><mo>)</mo></mrow></math></span>, which makes it possible to deal with some large clustering problem.</div></div>","PeriodicalId":50737,"journal":{"name":"Applied Soft Computing","volume":"167 ","pages":"Article 112432"},"PeriodicalIF":7.2,"publicationDate":"2024-10-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142654025","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Gesture recognition framework for upper-limb prosthetics using entropy features from electromyographic signals and a Gaussian kernel SVM classifier 使用肌电信号熵特征和高斯核 SVM 分类器的上肢假肢手势识别框架

IF 7.2 1区计算机科学

Applied Soft Computing Pub Date : 2024-10-30 DOI: 10.1016/j.asoc.2024.112382

Prabhavathy T. , Vinodh Kumar Elumalai , Balaji E.

{"title":"Gesture recognition framework for upper-limb prosthetics using entropy features from electromyographic signals and a Gaussian kernel SVM classifier","authors":"Prabhavathy T. , Vinodh Kumar Elumalai , Balaji E.","doi":"10.1016/j.asoc.2024.112382","DOIUrl":"10.1016/j.asoc.2024.112382","url":null,"abstract":"<div><div>This paper puts forward a novel entropy features based multi-class SVM classifier framework to predict the limb movement of the transradial amputees from the surface electromyography (sEMG) signals. The major challenges with the sEMG signal are nonlinear and non-stationary characteristics and susceptibility to noise. Consequently, a robust and an effective feature extraction framework which is invariant to force level variations is central in sEMG based prosthesis control. To address the aforementioned challenges, this study leverages the potential of variational mode decomposition (VMD) technique to identify the prominent frequency modes of the sEMG signals, and performs the spectral evaluation of the decomposed sEMG modes to identify the dominant ones to extract the entropy features. Subsequently, we evaluate the efficacy of four nonlinear optimal feature selection techniques and identify the prominent entropy features to train the multi-class SVM model that can predict the gestures. Specifically, to handle the nonlinearly separable input data, this study implements a kernelization named a radial basis function (RBF), which has good generalization and noise tolerance features. The efficacy of the proposed framework is tested using the publicly available datasets that contain gestures from transradial and congenital amputees for functional gestures. Experimental results obtained for various gestures with dynamic force levels underscore that the proposed framework is highly robust against the force level variations and can achieve a classification accuracy of 99.07%.</div></div>","PeriodicalId":50737,"journal":{"name":"Applied Soft Computing","volume":"167 ","pages":"Article 112382"},"PeriodicalIF":7.2,"publicationDate":"2024-10-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142656736","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Physics-informed neutral network with physically consistent and residual learning for excavator precision operation control 具有物理一致性和残差学习功能的物理信息中性网络，用于挖掘机精确操作控制

IF 7.2 1区计算机科学

Applied Soft Computing Pub Date : 2024-10-29 DOI: 10.1016/j.asoc.2024.112402

Chenlong Feng , Jixin Wang , Yuying Shen , Qi Wang , Yi Xiong , Xudong Zhang , Jiuchen Fan

{"title":"Physics-informed neutral network with physically consistent and residual learning for excavator precision operation control","authors":"Chenlong Feng , Jixin Wang , Yuying Shen , Qi Wang , Yi Xiong , Xudong Zhang , Jiuchen Fan","doi":"10.1016/j.asoc.2024.112402","DOIUrl":"10.1016/j.asoc.2024.112402","url":null,"abstract":"<div><div>The data-driven methodologies can establish accurate Inverse Dynamics Model (IDM) of the excavator thus improving control precisions. However, the inherent black-box nature of these models often results in overfitting to the dataset, leading to predictions that deviate from the constraints of physical system. Consequently, this can lead to controller failures, introducing unpredictable behavior that threatens operation precision. In addition, the uncertainty of the external disturbance poses great challenge to the precision of controller. This study presents a physics-informed neural network to build accurate IDM with physical consistency. The Rigid Body Dynamics (RBD) of the excavator are coupled within a Deep Lagrangian Network (DeLaN), while a Convolutional Neural Network (CNN) and a Long Short-Term Memory Network (LSTM) are employed to assimilate the residual nonlinear characteristics, such as hydraulic flexibilities and stick–slip friction. To the uncertainty of the external disturbance, the Prescribed Performance Inverse Dynamics Controller combination with the DeLaN-CNN-LSTM model (PPIDC-DCL) is constructed for precise control by constraining the control error within a finite region. The experimental results demonstrate that the model captures the underlying structure of the dynamic and builds the IDM with high accuracy and robustness. Moreover, the PPIDC-DCL controller effectively constrains the control error and realizes precision control. The proposed method has potential applications and provides novel insights for achieving precise operation control of excavators.</div></div>","PeriodicalId":50737,"journal":{"name":"Applied Soft Computing","volume":"167 ","pages":"Article 112402"},"PeriodicalIF":7.2,"publicationDate":"2024-10-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142573120","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

A MIL-based framework via contrastive instance learning and multimodal learning for long-term ECG classification 基于 MIL 的框架，通过对比实例学习和多模态学习实现长期心电图分类

IF 7.2 1区计算机科学

Applied Soft Computing Pub Date : 2024-10-29 DOI: 10.1016/j.asoc.2024.112372

Haozhan Han , Cheng Lian , Bingrong Xu , Zhigang Zeng , Adi Alhudhaif , Kemal Polat

{"title":"A MIL-based framework via contrastive instance learning and multimodal learning for long-term ECG classification","authors":"Haozhan Han , Cheng Lian , Bingrong Xu , Zhigang Zeng , Adi Alhudhaif , Kemal Polat","doi":"10.1016/j.asoc.2024.112372","DOIUrl":"10.1016/j.asoc.2024.112372","url":null,"abstract":"<div><div>Recently, deep learning-based models are widely employed for electrocardiogram (ECG) classification. However, classifying long-term ECGs, which contain vast amounts of data, is challenging. Due to the limitation of memory with respect to the original data size, preprocessing techniques such as resizing or cropping are often applied, leading to information loss. Therefore, introducing multi-instance learning (MIL) to address long-term ECG classification problems is crucial. However, a major drawback of employing MIL is the destruction of sample integrity, which consequently hinders the interaction among instances. To tackle this challenge, we proposed a multimodal MIL neural network named CIMIL, which consists of three key components: an instance interactor, a feature fusion method based on attention mechanisms, and a multimodal contrastive instance loss. First, we designed an instance interactor to improve the interaction and keep continuity among instances. Second, we proposed a novel feature fusion method based on attention mechanisms to effectively aggregate multimodal instance features for final classification, which selects key instances within each class, not only enhances the performance of our model but also reduces the number of parameters. Third, a multimodal contrastive instance loss is proposed to enhance the model’s ability to distinguish positive and negative multimodal instances. Finally, we evaluated CIMIL on both intrapatient and interpatient patterns of two commonly used ECG datasets. The experimental results show that the proposed CIMIL outperforms existing state-of-the-art methods on long-term ECG tasks.</div></div>","PeriodicalId":50737,"journal":{"name":"Applied Soft Computing","volume":"167 ","pages":"Article 112372"},"PeriodicalIF":7.2,"publicationDate":"2024-10-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142573149","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Learnable feature alignment with attention-based data augmentation for handling data issue in ancient documents 利用可学习的特征对齐和基于注意力的数据增强技术处理古文献中的数据问题

IF 7.2 1区计算机科学

Applied Soft Computing Pub Date : 2024-10-29 DOI: 10.1016/j.asoc.2024.112394

Amin Jalali , Sangbeom Lee , Minho Lee

{"title":"Learnable feature alignment with attention-based data augmentation for handling data issue in ancient documents","authors":"Amin Jalali , Sangbeom Lee , Minho Lee","doi":"10.1016/j.asoc.2024.112394","DOIUrl":"10.1016/j.asoc.2024.112394","url":null,"abstract":"<div><div>Recognizing ancient cursive handwritten characters presents unique challenges due to the diversity of writing styles and significant class imbalances, where some characters have disproportionately more samples than others. This imbalance leads to higher misclassification rates for minority classes compared to majority classes. To address these challenges, we propose a novel framework that integrates learnable channel and spatial attention modules to effectively align features between source and target domains for better representation. Our approach incorporates a learnable sequential feature alignment process that dynamically adjusts to the specific characteristics of the data, enhancing the transfer of knowledge across domains. Furthermore, we introduce an attention-based augmentation module to amplify the influence of tail classes. This module leverages class activation maps to identify and augment discriminative features, ensuring the model focuses on the most semantically rich regions, particularly for minority classes. As a result, it aligns the weight norms of minority classes with those of majority classes, effectively mitigating the limitations posed by imbalanced class distributions. This approach effectively mitigates the constraints posed by imbalanced character distributions in ancient handwritten documents. The proposed method increases the accuracy for the CCR, Hanja, Nancho, and Kuzushiji datasets.</div></div>","PeriodicalId":50737,"journal":{"name":"Applied Soft Computing","volume":"167 ","pages":"Article 112394"},"PeriodicalIF":7.2,"publicationDate":"2024-10-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142578775","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Effective Bi-decoding networks for rail-surface defect detection by knowledge distillation 通过知识提炼实现轨道表面缺陷检测的有效双解码网络

IF 7.2 1区计算机科学

Applied Soft Computing Pub Date : 2024-10-29 DOI: 10.1016/j.asoc.2024.112422

Wujie Zhou , Yue Wu , Weiwei Qiu , Caie Xu , Fangfang Qiang

{"title":"Effective Bi-decoding networks for rail-surface defect detection by knowledge distillation","authors":"Wujie Zhou , Yue Wu , Weiwei Qiu , Caie Xu , Fangfang Qiang","doi":"10.1016/j.asoc.2024.112422","DOIUrl":"10.1016/j.asoc.2024.112422","url":null,"abstract":"<div><div>No-service rail-surface defect detection is a crucial method for assessing the quality of railroad tracks. However, the low-contrast and dark-tone characteristics of track-surface textures pose challenges to current defect-monitoring techniques. Real-time and on-site online inspections are important to ensure safe railway operation; however, most complex models for no-service inspections are difficult to deploy on mobile devices. To address these challenges and overcome the detection difficulties associated with complex scenes, we designed a knowledge distillation-based double decoding-layer refinement network (EBDNet-KD). The first decoding process is guided by a bimodal high-level semantic feature map obtained by extending the attention-based graph convolution to incrementally enhance the dual-stream features and obtain an image restoration prior. A divide-and-conquer decoder is then designed to distinguish features using different decoding layers. The prior is then used in the second decoding layer, which enables the bimodal features to interact fully and obtain the final prediction map. We introduce a knowledge distillation strategy that enables a lightweight, compact student network to learn a complex teacher network’s feature extraction process. This facilitates pixel-consistent learning of the knowledge within the bi-decoder layer, as well as bidirectional learning of the focused contextual response knowledge to optimize the model. The EBDNet-KD significantly reduces computational costs while guaranteeing performance with a parameter count of only 28 M. EBDNet-KD demonstrated superior performance over 15 state-of-the-art methods in experiments conducted on NEU RSDDS-AUG, an industrial RGB-depth dataset. We assessed the generalizability of EBDNet-KD by evaluating its performance on three additional public datasets, yielding competitive results. The source code and results can be found at <span><span>https://github.com/Wuyue15/EBDNet</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":50737,"journal":{"name":"Applied Soft Computing","volume":"167 ","pages":"Article 112422"},"PeriodicalIF":7.2,"publicationDate":"2024-10-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142654121","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Improving policy training for autonomous driving through randomized ensembled double Q-learning with Transformer encoder feature evaluation 通过随机组合双 Q-learning 与 Transformer 编码器特征评估改进自动驾驶的策略训练

IF 7.2 1区计算机科学

Applied Soft Computing Pub Date : 2024-10-29 DOI: 10.1016/j.asoc.2024.112386

Jie Fan, Xudong Zhang, Yuan Zou, Yuanyuan Li, Yingqun Liu, Wenjing Sun

{"title":"Improving policy training for autonomous driving through randomized ensembled double Q-learning with Transformer encoder feature evaluation","authors":"Jie Fan, Xudong Zhang, Yuan Zou, Yuanyuan Li, Yingqun Liu, Wenjing Sun","doi":"10.1016/j.asoc.2024.112386","DOIUrl":"10.1016/j.asoc.2024.112386","url":null,"abstract":"<div><div>In the burgeoning field of autonomous driving, reinforcement learning (RL) has gained prominence for its adaptability and intelligent decision-making. However, conventional RL methods face challenges in efficiently extracting relevant features from high-dimensional inputs and maximizing the use of environment-agent interaction data. To surmount these obstacles, this paper introduces a novel RL-based approach that integrates randomized ensembled double Q-Learning (REDQ) with a Transformer encoder. The Transformer encoder’s attention mechanism is utilized to dynamically evaluate features according to their relevance in different driving scenarios. Simultaneously, the implementation of REDQ, characterized by a high update-to-data (UTD) ratio, enhances the utilization of interaction data during policy training. Especially, the incorporation of ensemble strategy and in-target minimization in REDQ significantly improves training stability, especially under high UTD conditions. Ablation studies indicate that the Transformer encoder exhibits significantly enhanced feature extraction capabilities compared to conventional network architectures, resulting in a 13.6% to 21.4% increase in success rate for the MetaDrive autonomous driving task. Additionally, when compared to standard RL methodologies, the proposed approach demonstrates a faster rate of reward acquisition and achieves a 67.5% to 69% improvement in success rate.</div></div>","PeriodicalId":50737,"journal":{"name":"Applied Soft Computing","volume":"167 ","pages":"Article 112386"},"PeriodicalIF":7.2,"publicationDate":"2024-10-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142578864","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0