Journal of King Saud University-Computer and Information Sciences最新文献

筛选
英文 中文
FuzzyTP-BERT: Enhancing extractive text summarization with fuzzy topic modeling and transformer networks FuzzyTP-BERT:利用模糊主题建模和转换器网络加强提取式文本摘要分析
IF 6.9 2区 计算机科学
Journal of King Saud University-Computer and Information Sciences Pub Date : 2024-06-07 DOI: 10.1016/j.jksuci.2024.102080
Aytuğ Onan , Hesham A. Alhumyani
{"title":"FuzzyTP-BERT: Enhancing extractive text summarization with fuzzy topic modeling and transformer networks","authors":"Aytuğ Onan ,&nbsp;Hesham A. Alhumyani","doi":"10.1016/j.jksuci.2024.102080","DOIUrl":"https://doi.org/10.1016/j.jksuci.2024.102080","url":null,"abstract":"<div><p>In the rapidly evolving field of natural language processing, the demand for efficient automated text summarization systems that not only distill extensive documents but also capture their nuanced thematic elements has never been greater. This paper introduces the FuzzyTP-BERT framework, a novel approach in extractive text summarization that synergistically combines Fuzzy Topic Modeling (FuzzyTM) with the advanced capabilities of Bidirectional Encoder Representations from Transformers (BERT). Unlike traditional extractive methods, FuzzyTP-BERT integrates fuzzy logic to refine topic modeling, enhancing the semantic sensitivity of summaries by allowing a more nuanced representation of word-topic relationships. This integration results in summaries that are not only coherent but also thematically rich, addressing a significant gap in current summarization technology. Extensive evaluations on benchmark datasets demonstrate that FuzzyTP-BERT significantly outperforms existing models in terms of ROUGE scores, effectively balancing topical relevance with semantic coherence. Our findings suggest that incorporating fuzzy logic into deep learning frameworks can markedly improve the quality of automated text summaries, potentially benefiting a wide range of applications in the information overload age.</p></div>","PeriodicalId":48547,"journal":{"name":"Journal of King Saud University-Computer and Information Sciences","volume":null,"pages":null},"PeriodicalIF":6.9,"publicationDate":"2024-06-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S1319157824001691/pdfft?md5=e14a922d86fdae0d4e0c5887e7adc430&pid=1-s2.0-S1319157824001691-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141313490","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
H2GCN: A hybrid hypergraph convolution network for skeleton-based action recognition H2GCN:基于骨骼的动作识别混合超图卷积网络
IF 6.9 2区 计算机科学
Journal of King Saud University-Computer and Information Sciences Pub Date : 2024-06-01 DOI: 10.1016/j.jksuci.2024.102072
Yiming Shao , Lintao Mao , Leixiong Ye , Jincheng Li , Ping Yang , Chengtao Ji , Zizhao Wu
{"title":"H2GCN: A hybrid hypergraph convolution network for skeleton-based action recognition","authors":"Yiming Shao ,&nbsp;Lintao Mao ,&nbsp;Leixiong Ye ,&nbsp;Jincheng Li ,&nbsp;Ping Yang ,&nbsp;Chengtao Ji ,&nbsp;Zizhao Wu","doi":"10.1016/j.jksuci.2024.102072","DOIUrl":"10.1016/j.jksuci.2024.102072","url":null,"abstract":"<div><p>Recent GCN-based works have achieved remarkable results for skeleton-based human action recognition. Nevertheless, while existing approaches extensively investigate pairwise joint relationships, only a limited number of models explore the intricate, high-order relationships among multiple joints. In this paper, we propose a novel hypergraph convolution method that represents the relationships among multiple joints with hyperedges, and dynamically refines the height-order relationship between hyperedges in the spatial, temporal, and channel dimensions. Specifically, our method initiates with a temporal-channel refinement hypergraph convolutional network, dynamically learning temporal and channel topologies in a data-dependent manner, which facilitates the capture of non-physical structural information inherent in the human body. Furthermore, to model various inter-joint relationships across spatio-temporal dimensions, we propose a spatio-temporal hypergraph joint module, which aims to encapsulate the dynamic spatial–temporal characteristics of the human body. Through the integration of these modules, our proposed model achieves state-of-the-art performance on RGB+D 60 and NTU RGB+D 120 datasets.</p></div>","PeriodicalId":48547,"journal":{"name":"Journal of King Saud University-Computer and Information Sciences","volume":null,"pages":null},"PeriodicalIF":6.9,"publicationDate":"2024-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S1319157824001617/pdfft?md5=fa620a352412bc3a5eed6aa760f3be55&pid=1-s2.0-S1319157824001617-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141132446","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Energy-aware task scheduling for streaming applications on NoC-based MPSoCs 基于 NoC 的 MPSoC 上流媒体应用的能量感知任务调度
IF 6.9 2区 计算机科学
Journal of King Saud University-Computer and Information Sciences Pub Date : 2024-06-01 DOI: 10.1016/j.jksuci.2024.102082
Suhaimi Abd Ishak , Hui Wu , Umair Ullah Tariq
{"title":"Energy-aware task scheduling for streaming applications on NoC-based MPSoCs","authors":"Suhaimi Abd Ishak ,&nbsp;Hui Wu ,&nbsp;Umair Ullah Tariq","doi":"10.1016/j.jksuci.2024.102082","DOIUrl":"https://doi.org/10.1016/j.jksuci.2024.102082","url":null,"abstract":"<div><p>Streaming applications are being extensively run on portable embedded systems, which are battery-operated and with limited memory. Thus, minimizing the total energy consumption of such a system is important. We investigate the problem of offline scheduling for streaming applications composed of non-preemptible periodic dependent tasks on homogeneous Network-on-Chip (NoC)-based Multiprocessor System-on-Chip (MPSoCs) such that their total energy consumption is minimized under memory constraints. We propose a novel unified approach that integrates task-level software pipelining with Dynamic Voltage and Frequency Scaling (DVFS) to solve the problem. Our approach is supported by a set of novel techniques, which include constructing an initial schedule based on a list scheduling where the priority of each task is its approximate successor-tree-consistent deadline such that the workload across all the processors is balanced, a retiming heuristic to transform intra-period dependencies into inter-period dependencies for enhancing parallelism, assigning an optimal discrete frequency for each task and each message using a Non-Linear Programming (NLP)-based algorithm and an Integer-Linear Programming (ILP)-based algorithm, and an incremental approach to reduce the memory usage of the retimed schedule in case of memory size violations. Using a set of real and synthetic benchmarks, we have implemented and compared our unified approach with two state-of-the-art approaches, RDAG+GeneS (<span>Wang et al., 2011</span>) , and JCCTS (<span>Wang et al., 2013a</span>). Experimental results show that our approach’s maximum, average, and minimum improvements over RDAG+GeneS (<span>Wang et al., 2011</span>) are 31.72%, 14.05%, and 7.00%, respectively. Our approach’s maximum, average, and minimum improvement over JCCTS (<span>Wang et al., 2013a</span>) are 35.58%, 17.04%, and 8.21%, respectively.</p></div>","PeriodicalId":48547,"journal":{"name":"Journal of King Saud University-Computer and Information Sciences","volume":null,"pages":null},"PeriodicalIF":6.9,"publicationDate":"2024-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S131915782400171X/pdfft?md5=9b5bfc05f1869e383654a6be6e74e93b&pid=1-s2.0-S131915782400171X-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141302800","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Automatic rumor recognition for public health and safety: A strategy combining topic classification and multi-dimensional feature fusion 为公共健康和安全自动识别谣言:主题分类与多维特征融合相结合的策略
IF 6.9 2区 计算机科学
Journal of King Saud University-Computer and Information Sciences Pub Date : 2024-06-01 DOI: 10.1016/j.jksuci.2024.102087
Yuxuan Zhang, Song Huang
{"title":"Automatic rumor recognition for public health and safety: A strategy combining topic classification and multi-dimensional feature fusion","authors":"Yuxuan Zhang,&nbsp;Song Huang","doi":"10.1016/j.jksuci.2024.102087","DOIUrl":"https://doi.org/10.1016/j.jksuci.2024.102087","url":null,"abstract":"<div><p>With the COVID-19 outbreak, health-related rumors have attracted significant attention from governments and global society. These rumors often mislead the public through multimedia, amplifying their negative impact and potentially manipulating public health narratives. On social media, detecting these rumors faces unique challenges, especially for emerging health events. Existing detection algorithms struggle because they mainly learn event-specific features that are not applicable to new or unseen events. To overcome this, we developed an end-to-end framework called the Health Domain Multimodal Rumor Detection Neural Network (HDRNN). This framework extracts invariant features and effectively detects new health-related rumors. It consists of three components: a multimodal feature extractor, a rumor detector, and an event discriminator. The multimodal feature extractor extracts text and visual features from posts, working with rumor detectors to learn discriminative features. Event discriminator remove specific features while retaining shared ones across events. Extensive experiments on datasets from Tencent News and Sina Weibo show that our HDRNN model excels in multimodal health rumor detection, surpassing current methods.</p></div>","PeriodicalId":48547,"journal":{"name":"Journal of King Saud University-Computer and Information Sciences","volume":null,"pages":null},"PeriodicalIF":6.9,"publicationDate":"2024-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S1319157824001769/pdfft?md5=1281c1b6c006ff6ac8e07b89476a3d71&pid=1-s2.0-S1319157824001769-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141294929","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Reversible data hiding based on global and local automatic contrast enhancement of low-light color images 基于低照度彩色图像全局和局部自动对比度增强的可逆数据隐藏技术
IF 6.9 2区 计算机科学
Journal of King Saud University-Computer and Information Sciences Pub Date : 2024-06-01 DOI: 10.1016/j.jksuci.2024.102076
Libo Han , Yanzhao Ren , Wanlin Gao , Xinfeng Zhang , Sha Tao
{"title":"Reversible data hiding based on global and local automatic contrast enhancement of low-light color images","authors":"Libo Han ,&nbsp;Yanzhao Ren ,&nbsp;Wanlin Gao ,&nbsp;Xinfeng Zhang ,&nbsp;Sha Tao","doi":"10.1016/j.jksuci.2024.102076","DOIUrl":"10.1016/j.jksuci.2024.102076","url":null,"abstract":"<div><p>Currently, many scholars have investigated reversible data hiding (RDH) based on automatic contrast enhancement (ACE). For RDH based on ACE (ACERDH), various methods have been proposed on how to better improve image quality. Preserving brightness can prevent the image from being over-enhanced. However, some ACERDH methods that can preserve brightness well cannot sufficiently enhance low-light images. Although some ACERDH methods that cannot preserve brightness well can more sufficiently enhance low-light color images, they lack the consideration of the local region, which is not conducive to enhancing the contrast of the local region. Therefore, a novel RDH method based on global ACE and local ACE is proposed. The global contrast is first improved by equalizing the global histogram. Then the high complexity region is optimized by the proposed RDH method based on double-layer ACE to further enhance the local contrast. The low-light color image can be well enhanced by applying the proposed method to enhance the R, G, and B channels sequentially. Experimental results demonstrated that the proposed method is superior to some existing advanced methods in obtaining better image quality and hiding more secret data.</p></div>","PeriodicalId":48547,"journal":{"name":"Journal of King Saud University-Computer and Information Sciences","volume":null,"pages":null},"PeriodicalIF":6.9,"publicationDate":"2024-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S1319157824001654/pdfft?md5=0dc4a7defc4e1cb9a6085b3ac7bb2b25&pid=1-s2.0-S1319157824001654-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141141098","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Improved lion swarm optimization algorithm to solve the multi-objective rescheduling of hybrid flowshop with limited buffer 用改进的狮群优化算法解决具有有限缓冲区的混合流水车间的多目标重新调度问题
IF 6.9 2区 计算机科学
Journal of King Saud University-Computer and Information Sciences Pub Date : 2024-06-01 DOI: 10.1016/j.jksuci.2024.102077
Tingyu Guan, Tingxin Wen, Bencong Kou
{"title":"Improved lion swarm optimization algorithm to solve the multi-objective rescheduling of hybrid flowshop with limited buffer","authors":"Tingyu Guan,&nbsp;Tingxin Wen,&nbsp;Bencong Kou","doi":"10.1016/j.jksuci.2024.102077","DOIUrl":"https://doi.org/10.1016/j.jksuci.2024.102077","url":null,"abstract":"<div><p>As the realities of production and operation in green and intelligent workshops become more variable, the adverse risks arising from disruptions to modernized workshop energy consumption schedules and customer churn caused by dynamic events are increasing. In order to solve those problems, we take the intelligent hybrid flow shop as the research subject, use buffer capacity and automated guided vehicles (AGVs) transport devices as resource constraints, construct a multi-objective rescheduling model that considers both energy consumption and customer satisfaction. According to the model characteristics, an improved lion swarm optimization algorithm (ILSO) is designed to solve the above model. To improve the initial solution quality and global search capability of the algorithm, ILSO is improved by combining the reverse learning initialization strategy of Logistic chaotic mapping with the tabu search strategy. The results of experiments on the proposed algorithm with different sizes of arithmetic cases and real cases in the workshop indicate that ILSO can effectively solve the bi-objective rescheduling problem oriented to inserting orders, and the proposed model can provide green dynamic scheduling solutions for manufacturing enterprises to achieve the purpose of transformation to green intelligent manufacturing.</p></div>","PeriodicalId":48547,"journal":{"name":"Journal of King Saud University-Computer and Information Sciences","volume":null,"pages":null},"PeriodicalIF":6.9,"publicationDate":"2024-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S1319157824001666/pdfft?md5=5d17fbfec8a74d4f04910f433ddf7824&pid=1-s2.0-S1319157824001666-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141250718","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Adaptive K values and training subsets selection for optimal K-NN performance on FPGA 自适应 K 值和训练子集选择,在 FPGA 上实现最佳 K-NN 性能
IF 6.9 2区 计算机科学
Journal of King Saud University-Computer and Information Sciences Pub Date : 2024-06-01 DOI: 10.1016/j.jksuci.2024.102081
Achraf El Bouazzaoui, Noura Jariri, Omar Mouhib, Abdelkader Hadjoudja
{"title":"Adaptive K values and training subsets selection for optimal K-NN performance on FPGA","authors":"Achraf El Bouazzaoui,&nbsp;Noura Jariri,&nbsp;Omar Mouhib,&nbsp;Abdelkader Hadjoudja","doi":"10.1016/j.jksuci.2024.102081","DOIUrl":"https://doi.org/10.1016/j.jksuci.2024.102081","url":null,"abstract":"<div><p>This study introduces an Adaptive K-Nearest Neighbors methodology designed for FPGA platforms, offering substantial improvements over traditional K-Nearest Neighbors implementations. By integrating a dynamic classifier selection system, our approach enhances adaptability, enabling on-the-fly adjustments of K values and subsets of training data. This flexibility results in up to a 10.66% improvement in accuracy and significantly reduces latency, rendering our system up to 3.918 times more efficient than conventional K-Nearest Neighbors techniques. The methodology’s efficacy is validated through experiments across multiple datasets, demonstrating its potential in optimizing both classification accuracy and system efficiency. The adaptive approach’s ability to improve response times, along with its flexibility, positions it as an ideal solution for real-time applications and highlights the advantages of the adaptive K-Nearest Neighbors methodology in overcoming the constraints of hardware-accelerated machine learning.</p></div>","PeriodicalId":48547,"journal":{"name":"Journal of King Saud University-Computer and Information Sciences","volume":null,"pages":null},"PeriodicalIF":6.9,"publicationDate":"2024-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S1319157824001708/pdfft?md5=fe96192276397d86b55be1cb1f47d494&pid=1-s2.0-S1319157824001708-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141250787","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Predicting DNA sequence splice site based on graph convolutional network and DNA graph construction 基于图卷积网络和 DNA 图构建预测 DNA 序列剪接位点
IF 6.9 2区 计算机科学
Journal of King Saud University-Computer and Information Sciences Pub Date : 2024-06-01 DOI: 10.1016/j.jksuci.2024.102089
Luo Rentao, Li Yelin, Guan Lixin, Li Mengshan
{"title":"Predicting DNA sequence splice site based on graph convolutional network and DNA graph construction","authors":"Luo Rentao,&nbsp;Li Yelin,&nbsp;Guan Lixin,&nbsp;Li Mengshan","doi":"10.1016/j.jksuci.2024.102089","DOIUrl":"https://doi.org/10.1016/j.jksuci.2024.102089","url":null,"abstract":"<div><p>Identifying splice sites is essential for gene structure analysis and eukaryotic genome annotation. Recently, computational and deep learning approaches for splice site detection have advanced, focusing on reducing false positives by distinguishing true from pseudo splice sites. This paper introduces GraphSplice, a method using graph convolutional neural networks. It encodes DNA sequences into directed graphs to extract features and predict splice sites. Tested across multiple datasets, GraphSplice consistently achieved high accuracy (91%-94%) and F1Scores (92%-94%), outperforming state-of-the-art models by up to 9.16% for donors and 5.64% for acceptors. Cross-species experiments also show GraphSplice’s capability to annotate splice sites in under-trained genomic datasets, proving its wide applicability as a tool for DNA splice site analysis.</p></div>","PeriodicalId":48547,"journal":{"name":"Journal of King Saud University-Computer and Information Sciences","volume":null,"pages":null},"PeriodicalIF":6.9,"publicationDate":"2024-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S1319157824001782/pdfft?md5=a168124b3f808fa8741574d862f7a5a1&pid=1-s2.0-S1319157824001782-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141302799","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
MRME-Net: Towards multi-semantics learning and long-tail problem of efficient event detection from social messages MRME-Net:从社交信息中高效检测事件的多语义学习和长尾问题
IF 6.9 2区 计算机科学
Journal of King Saud University-Computer and Information Sciences Pub Date : 2024-06-01 DOI: 10.1016/j.jksuci.2024.102070
Ruihan Wu , Tianfa Hong , FangYing Wan
{"title":"MRME-Net: Towards multi-semantics learning and long-tail problem of efficient event detection from social messages","authors":"Ruihan Wu ,&nbsp;Tianfa Hong ,&nbsp;FangYing Wan","doi":"10.1016/j.jksuci.2024.102070","DOIUrl":"https://doi.org/10.1016/j.jksuci.2024.102070","url":null,"abstract":"<div><p>Discovering trending social events (e.g., major meetings, political scandals, natural disasters, etc.) from social messages is vital because it emphasizes important events and can help people comprehend the world. However, the heterogeneous semantics enrichment, severe long-tail problems, and sparse text contents of social messages pose great challenges to event detection, often leading to limited generalization ability and accuracy. In this paper, we propose a novel Multi-Relational Meta-Enhanced Network (MRME-Net) architecture to learn social events. First, we model social messages into a multi-relational message graph, incorporating abundant meta-semantics along with various meta-relations. Second, we present a multi-relational graph attention network based on Sophia by using a dual-step message aggregation mechanisms to capture the local features of neighboring messages and global semantics of mutiple relations and ultimately learn social message embeddings. We use Sophia optimizer to reduce the massive time and cost of training. Third, in order to address the long-tail problem, we introduce a locally-adapted meta-learning framework in social event detection for the first time and propose a novel META-TAILENH embedding enhancement strategy to refine tail node embeddings in multi-relational graph. Eventually, we conduct the detection of social events according to the hierarchical clustering algorithm. Extensive experiments have been carried out to evaluate MRME-Net on the MAVEN and Twitter dataset, revealing a notable improvement of 3 %–13 %, 4 %–20 % and 6 %–30 % increases on NMI, AMI and ARI in the social event detection task.</p></div>","PeriodicalId":48547,"journal":{"name":"Journal of King Saud University-Computer and Information Sciences","volume":null,"pages":null},"PeriodicalIF":6.9,"publicationDate":"2024-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S1319157824001599/pdfft?md5=16a8646203ecb19a3bedc076ec6e6c1e&pid=1-s2.0-S1319157824001599-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141241979","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A high speed inference architecture for multimodal emotion recognition based on sparse cross modal encoder 基于稀疏交叉模态编码器的多模态情感识别高速推理架构
IF 6.9 2区 计算机科学
Journal of King Saud University-Computer and Information Sciences Pub Date : 2024-06-01 DOI: 10.1016/j.jksuci.2024.102092
Lin Cui, Yuanbang Zhang, Yingkai Cui, Boyan Wang, Xiaodong Sun
{"title":"A high speed inference architecture for multimodal emotion recognition based on sparse cross modal encoder","authors":"Lin Cui,&nbsp;Yuanbang Zhang,&nbsp;Yingkai Cui,&nbsp;Boyan Wang,&nbsp;Xiaodong Sun","doi":"10.1016/j.jksuci.2024.102092","DOIUrl":"https://doi.org/10.1016/j.jksuci.2024.102092","url":null,"abstract":"<div><p>In recent years, multimodal emotion recognition models are using pre-trained networks and attention mechanisms to pursue higher accuracy, which increases the training burden and slows down the training and inference speed. In order to strike a balance between speed and accuracy, this paper proposes a speed-optimized multimodal emotion recognition architecture for speech and text emotion recognition. In the feature extraction part, a lightweight residual graph convolutional network (ResGCN) is selected as the speech feature extractor, and an efficient RoBERTa pre-trained network is used as the text feature extractor. Then, an algorithm complexity-optimized sparse cross-modal encoder (SCME) is proposed and used to fuse these two types of features. Finally, a new gated fusion module (GF) is used to weight multiple results and input them into a fully connected layer (FC) for classification. The proposed method is tested on the IEMOCAP dataset and the MELD dataset, achieving weighted accuracies (WA) of 82.4% and 65.0%, respectively. This method achieves higher accuracy than the listed methods while having an acceptable training and inference speed.</p></div>","PeriodicalId":48547,"journal":{"name":"Journal of King Saud University-Computer and Information Sciences","volume":null,"pages":null},"PeriodicalIF":6.9,"publicationDate":"2024-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.sciencedirect.com/science/article/pii/S1319157824001812/pdfft?md5=b0fe2e31975a2a4019a33870a9ba1e11&pid=1-s2.0-S1319157824001812-main.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141302801","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
相关产品
×
本文献相关产品
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信