Expert Systems with Applications最新文献

筛选
英文 中文
OpenHRD: Hierarchical representation decoupling for open-world semi-supervised learning 开放世界半监督学习的分层表示解耦
IF 7.5 1区 计算机科学
Expert Systems with Applications Pub Date : 2025-04-21 DOI: 10.1016/j.eswa.2025.127695
Guanjia Zhang, Weiwei Xing, Qiyue Liang, Xiaoyu Guo, Xiang Wei, Jian Zhang
{"title":"OpenHRD: Hierarchical representation decoupling for open-world semi-supervised learning","authors":"Guanjia Zhang,&nbsp;Weiwei Xing,&nbsp;Qiyue Liang,&nbsp;Xiaoyu Guo,&nbsp;Xiang Wei,&nbsp;Jian Zhang","doi":"10.1016/j.eswa.2025.127695","DOIUrl":"10.1016/j.eswa.2025.127695","url":null,"abstract":"<div><div>In realistic open-world semi-supervised scenarios, novel classes always emerge from unlabeled data, which leads to the performance degradation of existing semi-supervised learning (SSL) methods. The absence of any supervisory signals for novel classes hinders the model from learning disentangled representations, causing the model to confuse known classes with novel classes and generate significant prediction bias towards known classes. In this paper, we propose a hierarchical representation decoupling approach, named OpenHRD, which jointly decouples representations at the instance level and class level for different samples to address this challenge. Specifically, at the instance level, we impose representation constraints on the most similar instance pairs with highest representation similarity to mitigate representation confusion between samples. Furthermore, we also propose an adaptive pseudo-label debiasing regularization method for unlabeled instances at the instance level, which effectively alleviate the prediction bias toward known classes during the model training. At the class level, we introduce an inter-class contrastive learning strategy for novel classes to enlarge the representation distinction between each novel class and other classes. Extensive experimental results on various settings over CIFAR-10, CIFAR-100, and ImageNet-100 demonstrate the superior performance of the proposed OpenHRD. We will release the code at: <span><span>https://github.com/srxhlife/OpenHRD</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":50461,"journal":{"name":"Expert Systems with Applications","volume":"282 ","pages":"Article 127695"},"PeriodicalIF":7.5,"publicationDate":"2025-04-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143869514","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
CELLMEA:A Collaboratively Enhanced Large Language Model-based Entity Alignment for aircraft fault maintenance CELLMEA:一种协同增强的基于大语言模型的飞机故障维修实体对齐方法
IF 7.5 1区 计算机科学
Expert Systems with Applications Pub Date : 2025-04-21 DOI: 10.1016/j.eswa.2025.127630
Xiangzhen Meng , Xiaoxuan Jiao , Jiahui Li , Shenglong Wang , Jinxin Pan , Bo Jing , Xilang Tang
{"title":"CELLMEA:A Collaboratively Enhanced Large Language Model-based Entity Alignment for aircraft fault maintenance","authors":"Xiangzhen Meng ,&nbsp;Xiaoxuan Jiao ,&nbsp;Jiahui Li ,&nbsp;Shenglong Wang ,&nbsp;Jinxin Pan ,&nbsp;Bo Jing ,&nbsp;Xilang Tang","doi":"10.1016/j.eswa.2025.127630","DOIUrl":"10.1016/j.eswa.2025.127630","url":null,"abstract":"<div><div>Aircraft fault knowledge graphs serve as a critical knowledge base for the intelligent maintenance and operations of aviation equipment. However, the entity alignment tasks in their construction remain overly dependent on manual annotation, leading to issues such as inconsistent annotation quality and low annotation efficiency. Unsupervised methods provide a promising solution and have garnered significant research interest. However, existing unsupervised entity alignment approaches often overlook the impact of noisy entities, presenting a significant challenge for aligning entities in aviation fault data. This paper proposes a solution by incorporating a large language model (LLM) into the entity alignment process for aircraft fault knowledge graphs. By leveraging the world knowledge encoded in the LLM, the approach enhances the performance of unsupervised entity alignment models. Specifically, we introduce the Collaboratively Enhanced-based Large Language Model Entity Alignment (CELLMEA), which utilizes data from the aircraft flight control system manual, fault analysis manual, and typical fault cases. The model’s architecture includes a multi-view semantic information embedding that integrates structural, relational, and semantic data. Additionally, we propose an adaptive method for mixing hard negative samples, which generates higher-quality negative entities by combining noisy negative samples with reliable ones. Furthermore, an incremental consistency regularization technique is introduced to progressively refine the robustness of pseudo-labeling within the CELLMEA model. Finally, experimental results on a flight control system entity alignment dataset demonstrate that CELLMEA outperforms all baseline models, achieving an MRR (Mean Reciprocal Rank) value of 0.917 ± 0.011. These results validate the model’s effectiveness in handling unlabeled data and lay the groundwork for the engineering of aircraft fault knowledge graphs.</div></div>","PeriodicalId":50461,"journal":{"name":"Expert Systems with Applications","volume":"282 ","pages":"Article 127630"},"PeriodicalIF":7.5,"publicationDate":"2025-04-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143869518","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
The unsupervised short text classification method based on GCN encoder–decoder and local enhancement 基于GCN编解码器和局部增强的无监督短文本分类方法
IF 7.5 1区 计算机科学
Expert Systems with Applications Pub Date : 2025-04-21 DOI: 10.1016/j.eswa.2025.127678
Yingying Wei , Ze Wang , Jianbin Li , Tao Li
{"title":"The unsupervised short text classification method based on GCN encoder–decoder and local enhancement","authors":"Yingying Wei ,&nbsp;Ze Wang ,&nbsp;Jianbin Li ,&nbsp;Tao Li","doi":"10.1016/j.eswa.2025.127678","DOIUrl":"10.1016/j.eswa.2025.127678","url":null,"abstract":"<div><div>Like all fields of data science, short text classification seeks to achieve high-quality results with limited data. Although supervised learning methods have made notable progress in this area, they require much-labeled data to achieve adequate accuracy. However, in many practical applications, labeled data is scarce, and manual labeling is not only time-consuming and labor-intensive but also expensive and may require specialized expertise. Therefore, this paper addresses the challenge of insufficient labeled data through unsupervised methods while ensuring the effective extraction of semantic features from the text. Building on this objective, we propose a novel unsupervised short text classification method within the framework of autoencoders. Specifically, we first design the MRFasGCN encoder and derive the relationships between nodes in its hidden layers, thereby enhancing the capture of text features and semantic information. Furthermore, we construct a dual-node-based decoder that reconstructs the topology and node attributes unsupervised. This approach compensates for feature deficiencies from multiple perspectives, alleviating the issue of insufficient features in short texts. Finally, we propose a localized enhancement method that integrates node features and topology, strengthening the connections between relevant nodes. This improves the model’s understanding of the text’s local context while mitigating the overfitting issues caused by feature sparsity in short texts. Extensive experimental results demonstrate the pronounced superiority of our proposed UEDE model over existing methods on the dataset, validating its effectiveness in short-text classification. Our code is submitted in <span><span>https://github.com/w123yy/UEDE</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":50461,"journal":{"name":"Expert Systems with Applications","volume":"282 ","pages":"Article 127678"},"PeriodicalIF":7.5,"publicationDate":"2025-04-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143859169","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
FeatureX: An explainable feature selection for deep learning FeatureX:用于深度学习的可解释特征选择
IF 7.5 1区 计算机科学
Expert Systems with Applications Pub Date : 2025-04-21 DOI: 10.1016/j.eswa.2025.127675
Siyi Liang , Yang Zhang , Kun Zheng, Yu Bai
{"title":"FeatureX: An explainable feature selection for deep learning","authors":"Siyi Liang ,&nbsp;Yang Zhang ,&nbsp;Kun Zheng,&nbsp;Yu Bai","doi":"10.1016/j.eswa.2025.127675","DOIUrl":"10.1016/j.eswa.2025.127675","url":null,"abstract":"<div><div>Feature selection is critical for the performance of deep learning models by reducing the dimensionality of feature sets to understand the features’ importance. Existing techniques focus on the statistical characteristics of different features, which makes them hard to understand due to complicated mathematical reasoning. Furthermore, feature selection can be impacted by model preferences, resulting in a lack of explainability. To this end, this paper proposes an effective method called FeatureX to obtain the optimal feature subset and enhance the explainability of the feature selection process through quantitative evaluation. Firstly, FeatureX proposes importance analysis to quantify the contribution of each feature to the deep learning model by leveraging feature perturbation. Secondly, to mitigate the multicollinearity, FeatureX employs statistical analysis to calculate the correlation coefficients of these features and removes redundant features based on the magnitude of the correlation coefficients. Finally, with the feature contribution and correlation coefficients, FeatureX screens these features automatically to identify the most relevant and high-contribution features. Based on existing research and prior knowledge of the data, FeatureX presets the values of relevant parameters and demonstrates their effectiveness through parameter sensitivity analysis. FeatureX is evaluated on 17 public datasets with 5 fundamental deep learning models. Experimental results show that FeatureX can reduce the number of features by an average of 47.83% and the accuracy of 63.33% deep learning models are improved. Furthermore, when comparing against the existing feature selection techniques, FeatureX improves the F-measure by an average of 1.61%, demonstrating its effectiveness.</div></div>","PeriodicalId":50461,"journal":{"name":"Expert Systems with Applications","volume":"282 ","pages":"Article 127675"},"PeriodicalIF":7.5,"publicationDate":"2025-04-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143864801","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Enhanced quantum long short-term memory neural network based multi-task learning for sentimental analysis and cyberbullying detection 基于增强量子长短期记忆神经网络的多任务学习情感分析和网络欺凌检测
IF 7.5 1区 计算机科学
Expert Systems with Applications Pub Date : 2025-04-21 DOI: 10.1016/j.eswa.2025.127555
K. Subhashree , S.Manoj Kumar
{"title":"Enhanced quantum long short-term memory neural network based multi-task learning for sentimental analysis and cyberbullying detection","authors":"K. Subhashree ,&nbsp;S.Manoj Kumar","doi":"10.1016/j.eswa.2025.127555","DOIUrl":"10.1016/j.eswa.2025.127555","url":null,"abstract":"<div><div>Increasing usage of social media by individuals led to a significant rise in cyberbullying. Detecting sarcasm is challenging because many comments contain sarcasm or aggressive language. Text sentiment classification helps in the identification of abusive words using some beneficial features. Several machine learning algorithms are used in the detection of cyberbullying by using natural language processing mechanism. However, Deep Learning (DL) algorithms provides significant improvement in outcomes due to various reasons such as effectively segments text and image data, handling of large dataset, automatic extraction of features. Hence, a novel DL method Hybrid averaged and weighted averaged review vector Quantum long short-term memory neural based Multi-task Learning with Black-winged kite Optimization (HQMLBO) is proposed. Pre-processing is performed to clean the raw data. Next, features are extracted using hybrid multi-scale with hash vectorization, and relevant features are selected via the hybrid pine cone geyser-inspired optimization algorithm. Finally, sentiment classification and cyberbullying detection are performed using HQMLBO. Various DL methods are analysed and compared over three datasets using Python software. The proposed model outperforms existing methods in terms of accuracy of 95.68% for internet movie database, 92.5% for yelp polarity and 97.86% for cyberbullying classification dataset.</div></div>","PeriodicalId":50461,"journal":{"name":"Expert Systems with Applications","volume":"282 ","pages":"Article 127555"},"PeriodicalIF":7.5,"publicationDate":"2025-04-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143876890","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
An intelligent and uncertain optimization framework for water-nitrogen synergistic management under extreme supply and demand water risks 极端供需水风险下水氮协同管理的智能不确定优化框架
IF 7.5 1区 计算机科学
Expert Systems with Applications Pub Date : 2025-04-21 DOI: 10.1016/j.eswa.2025.127829
Xianghui Xu , Yaowen Xu , Yan Zhou , Rentao Li , Yijia Wang , Hongda Lian , Yingshan Chen , Zhengwei Zhang , Mo Li
{"title":"An intelligent and uncertain optimization framework for water-nitrogen synergistic management under extreme supply and demand water risks","authors":"Xianghui Xu ,&nbsp;Yaowen Xu ,&nbsp;Yan Zhou ,&nbsp;Rentao Li ,&nbsp;Yijia Wang ,&nbsp;Hongda Lian ,&nbsp;Yingshan Chen ,&nbsp;Zhengwei Zhang ,&nbsp;Mo Li","doi":"10.1016/j.eswa.2025.127829","DOIUrl":"10.1016/j.eswa.2025.127829","url":null,"abstract":"<div><div>Extreme climate conditions, such as droughts and floods, are becoming more frequent, prolonged, and severe due to global warming. The multiscenario water–nitrogen resource allocation program (WNRAP) not only addresses the risks associated with uncertainty in water supply and demand in extreme climates but also reduces the likelihood of these extreme events by decreasing carbon emissions. Therefore, to increase the ability of agricultural water–nitrogen management systems (AWNMS) to cope with extreme hydrological events, a multidimensional uncertainty model combined with an intelligent optimization framework that integrates the R-vine copula and interval two-stage stochastic programming (RITSP-IGWO) was developed. First, the R-vine copula model was used to characterize the non-Gaussian correlations among rainfall, runoff, and crop actual evapotranspiration (ET<sub>c,act</sub>). Second, the interval two-stage stochastic programming (ITSP) model was employed to address uncertainties in the environmental parameters. Finally, the multistrategy improved gray wolf optimization (IGWO) algorithm was utilized to solve the ITSP model and obtain the WNRAP results for 27 scenarios. The optimization results revealed that the water and nitrogen fertilizer use efficiencies in the WNRAP increased by 18.69% and 21.83%, respectively, whereas the CO<sub>2</sub> emissions decreased by 11.63%, and the solution efficiency improved by 67.99%. This framework can generate accurate and robust WNRAPs, providing effective theoretical support and practical guidance for sustainable agriculture to address the risks of extreme climate uncertainty.</div></div>","PeriodicalId":50461,"journal":{"name":"Expert Systems with Applications","volume":"282 ","pages":"Article 127829"},"PeriodicalIF":7.5,"publicationDate":"2025-04-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143869641","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
PFGRS: A Privacy-preserving Subgraph-level Federated Graph learning for Recommender System PFGRS:用于推荐系统的隐私保护子图层联合图学习
IF 7.5 1区 计算机科学
Expert Systems with Applications Pub Date : 2025-04-20 DOI: 10.1016/j.eswa.2025.127615
Qingqiang Qi, Chengyu Hu, Tongyaqi Li, Peng Tang, Shanqing Guo
{"title":"PFGRS: A Privacy-preserving Subgraph-level Federated Graph learning for Recommender System","authors":"Qingqiang Qi,&nbsp;Chengyu Hu,&nbsp;Tongyaqi Li,&nbsp;Peng Tang,&nbsp;Shanqing Guo","doi":"10.1016/j.eswa.2025.127615","DOIUrl":"10.1016/j.eswa.2025.127615","url":null,"abstract":"<div><div>The federated graph recommender system has garnered significant attention due to its broad applicability and the capabilities of Graph Neural Networks. Addressing the challenges of non-independent and identically distributed (Non-IID) data, along with ensuring privacy while achieving nodes’ features sharing among clients, is pivotal in federated graph recommender systems. In this study, we introduce the Deep neural network-based Graph Convolutional Collaborative Filtering model (DGCF), comprising two key modules: the DEEP module and the GCN module. The DEEP module, a feedforward neural network, can learn high-order feature interactions within users and items, while the GCN module can capture collaborative signals. Building upon DGCF, we propose a Privacy-preserving Subgraph-level Federated Graph Learning for Recommender System (PFGRS). To mitigate the Non-IID problem and extend the local graph for each client, PFGRS leverages differential privacy and trusted execution environments, which avoids the introduction of third-party servers and local differential privacy. More importantly, by segregating local training into independent training and extended training, PFGRS enables nodes’ feature to be shared indirectly among clients in federated learning. We conduct comprehensive experiments on real datasets. The experimental results not only show the superior performance of DGCF but also well demonstrate the significant effectiveness of PFGRS.</div></div>","PeriodicalId":50461,"journal":{"name":"Expert Systems with Applications","volume":"282 ","pages":"Article 127615"},"PeriodicalIF":7.5,"publicationDate":"2025-04-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143864797","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A multi-timescale dynamic graph attention network (MTDGAT) for short-term traffic prediction under special events 基于多时间尺度的动态图关注网络(MTDGAT)在特殊事件下的短期交通预测
IF 7.5 1区 计算机科学
Expert Systems with Applications Pub Date : 2025-04-19 DOI: 10.1016/j.eswa.2025.127649
Tian Lei, Yuxin Ding, Jingpeng Wen, Xiaohong Yin, Lei Gong, Qin Luo
{"title":"A multi-timescale dynamic graph attention network (MTDGAT) for short-term traffic prediction under special events","authors":"Tian Lei,&nbsp;Yuxin Ding,&nbsp;Jingpeng Wen,&nbsp;Xiaohong Yin,&nbsp;Lei Gong,&nbsp;Qin Luo","doi":"10.1016/j.eswa.2025.127649","DOIUrl":"10.1016/j.eswa.2025.127649","url":null,"abstract":"<div><div>Accurate traffic prediction is a key task in Intelligent Transportation Systems (ITS) and is crucial for proactive traffic control and management. Traffic prediction under special events (SEs) has long been challenging due to the uneven spatio-temporal distribution of traffic state and high traffic fluctuations. The present work proposes a Multi-Timescale Dynamic Graph Attention Network(MTDGAT) for short-term traffic prediction under SEs. Specifically, we design a multi-timescale block that employs attention mechanisms to model the relationships between historical traffic states and future traffic states across different time scales in a fine-grained manner, aiming to accurately capture the complex traffic evolution patterns under SEs. Our model adopts an encoder–decoder architecture, wherein the encoder combines historical data and SEs Encoding to construct a historical multi-timescale spatio-temporal graph, which is then transformed into a future multi-timescale spatio-temporal graph through a transform module. Extensive experiments were conducted on real-world traffic datasets to further validate the performance of the proposed model. First, ablation experiment results demonstrate the superiority of the proposed MTDGAT model in capturing short-term evolution of traffic states under SEs. Furthermore, through comprehensive comparison experiments, it is indicated that the MTDGAT outperforms other baseline models across all prediction steps under SEs. Specifically, MTDGAT achieves MAE of 3.21 and MAPE of 14.46 for 1-step prediction, alongside MAE of 4.15 and MAPE of 19.9 in 4-step prediction. The outcomes of the present work could provide deeper insights into spatio-temporal traffic evolution under the influence of SEs and contribute to the formulation of proactive traffic management and control strategies in such scenarios.</div></div>","PeriodicalId":50461,"journal":{"name":"Expert Systems with Applications","volume":"282 ","pages":"Article 127649"},"PeriodicalIF":7.5,"publicationDate":"2025-04-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143864811","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Pruning remote photoplethysmography networks using weight-gradient joint criterion 利用权重梯度联合准则修剪远程光容积脉搏波网络
IF 7.5 1区 计算机科学
Expert Systems with Applications Pub Date : 2025-04-19 DOI: 10.1016/j.eswa.2025.127623
Changchen Zhao , Shunhao Zhang , Pengcheng Cao , Shichao Cheng , Jianhai Zhang
{"title":"Pruning remote photoplethysmography networks using weight-gradient joint criterion","authors":"Changchen Zhao ,&nbsp;Shunhao Zhang ,&nbsp;Pengcheng Cao ,&nbsp;Shichao Cheng ,&nbsp;Jianhai Zhang","doi":"10.1016/j.eswa.2025.127623","DOIUrl":"10.1016/j.eswa.2025.127623","url":null,"abstract":"<div><div>With the rapid advancement of remote photoplethysmography (rPPG), there is an urgent need to deploy rPPG algorithms on edge devices for efficient and accurate inference. However, due to limited computational resources, many rPPG neural networks require tailoring before they can be applied to these devices. Most existing network pruning algorithms rely on a single indicator to measure the importance of a connection, often resulting in the premature removal of crucial connections during the early stages of training. In this paper, we propose a novel pruning scheme that jointly considers the weight and gradient of a connection as the importance metric, while also taking into account the dynamics of the connection during the training process. Specifically, connections with large weights and small gradients are identified as stable and important, and should be retained. Secondly, connections with small weights and large gradients, although potentially significant for development, are likely to be removed but should be allowed to regenerate. Additionally, connections with small weights and small gradients, which are stable and necessary, are also considered. An importance indicator is designed for each of these three types of connections and is utilized in the drop, regenerate, and trim steps, respectively. The proposed pruning scheme is evaluated on two existing networks (DeeprPPG and PhysNet) using the PURE dataset. The results demonstrate that our approach possesses smaller network sparsity, fewer parameters, and fewer floating-point operations (FLOPs) to achieve a given level of accuracy compared to existing pruning methods. This study validates the feasibility of fine-grained pruning for small networks and highlights the effectiveness of considering the dynamics of connections during the training process.</div></div>","PeriodicalId":50461,"journal":{"name":"Expert Systems with Applications","volume":"282 ","pages":"Article 127623"},"PeriodicalIF":7.5,"publicationDate":"2025-04-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143859165","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
TaiChiNet: PCA-based Ying-Yang dilution of inter- and intra-BERT layers to represent anti-coronavirus peptides 基于pca的bert层间和层内的阴阳稀释来代表抗冠状病毒肽
IF 7.5 1区 计算机科学
Expert Systems with Applications Pub Date : 2025-04-19 DOI: 10.1016/j.eswa.2025.127786
Kewei Li , Shiying Ding , Zhe Guo , Yusi Fan , Hongmei Liu , Yannan Sun , Gongyou Zhang , Ruochi Zhang , Lan Huang , Fengfeng Zhou
{"title":"TaiChiNet: PCA-based Ying-Yang dilution of inter- and intra-BERT layers to represent anti-coronavirus peptides","authors":"Kewei Li ,&nbsp;Shiying Ding ,&nbsp;Zhe Guo ,&nbsp;Yusi Fan ,&nbsp;Hongmei Liu ,&nbsp;Yannan Sun ,&nbsp;Gongyou Zhang ,&nbsp;Ruochi Zhang ,&nbsp;Lan Huang ,&nbsp;Fengfeng Zhou","doi":"10.1016/j.eswa.2025.127786","DOIUrl":"10.1016/j.eswa.2025.127786","url":null,"abstract":"<div><div>Numerous studies have demonstrated that biological sequences, such as DNA, RNA, and peptide, can be considered the “language of life”. Utilizing pre-trained language models (LMs) like ESM2, GPT, and BERT have yielded state-of-the-art (SOTA) results in many cases. However, the increasing size of datasets exponentially escalates the time and hardware resources required for fine-tuning a complete LM. This paper assumed that natural language shared linguistic logic with the “language of life” like peptides. We took the LM BERT model as an example in a novel Principal Component Analysis (PCA)-based Ying-Yang dilution network of the inter- and intra-BERT layers, termed TaiChiNet, for feature representation of peptide sequences. The Ying-Yang dilution architecture fuses the PCA transformation matrices trained on positive and negative samples, respectively. We transferred the TaiChiNet features into a subtractive layer feature space and observed that TaiChiNet just rotated the original subtractive features with a certain angle and didn’t change the relative distance among the dimensions. TaiChiNet-engineered features together with the hand-crafted (HC) ones were integrated for the prediction model of anti-coronavirus peptides (TaiChiACVP). Experimental results demonstrated that the TaiChiACVP model achieved new SOTA performance and remarkably short training time on five imbalanced datasets established for the anti-coronavirus peptide (ACVP) prediction task. The decision paths of the random forest classifier illustrated that TaiChiNet features can complement HC features for better decisions. TaiChiNet has also learned the latent features significantly correlated with physicochemical properties including molecular weight. This makes an explainable connection between the deep learning-represented features and the ACVP-associated physicochemical properties. Additionally, we extended our work to the other LMs, including ESM2 with 6 and 12 layers, ProGen2 small and base version, ProtBERT, and ProtGPT2. Due to the limitations of these recent LMs, none of them outperforms TaiChiACVP. However, some limitations of TaiChiNet remained to be investigated in the future, including learnable rotation degrees, extended fusions of more layers, and end-to-end training architecture. The source code is freely available at: <span><span>http://www.healthinformaticslab.org/supp/resources.php</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":50461,"journal":{"name":"Expert Systems with Applications","volume":"282 ","pages":"Article 127786"},"PeriodicalIF":7.5,"publicationDate":"2025-04-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"143869640","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
相关产品
×
本文献相关产品
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信