A Bi-Directional GRU Architecture for the Self-Attention Mechanism: An Adaptable, Multi-Layered Approach with Blend of Word Embedding

IF 1.3 Q3 ENGINEERING, MULTIDISCIPLINARY
Amit Pimpalkar, Jeberson Retna Raj
{"title":"A Bi-Directional GRU Architecture for the Self-Attention Mechanism: An Adaptable, Multi-Layered Approach with Blend of Word Embedding","authors":"Amit Pimpalkar, Jeberson Retna Raj","doi":"10.46604/ijeti.2023.11510","DOIUrl":null,"url":null,"abstract":"Sentiment analysis (SA) has become an essential component of natural language processing (NLP) with numerous practical applications to understanding “what other people think”. Various techniques have been developed to tackle SA using deep learning (DL); however, current research lacks comprehensive strategies incorporating multiple-word embeddings. This study proposes a self-attention mechanism that leverages DL and involves the contextual integration of word embedding with a time-dispersed bidirectional gated recurrent unit (Bi-GRU). This work employs word embedding approaches GloVe, word2vec, and fastText to achieve better predictive capabilities. By integrating these techniques, the study aims to improve the classifier’s capability to precisely analyze and categorize sentiments in textual data from the domain of movies. The investigation seeks to enhance the classifier’s performance in NLP tasks by addressing the challenges of underfitting and overfitting in DL. To evaluate the model’s effectiveness, an openly available IMDb dataset was utilized, achieving a remarkable testing accuracy of 99.70%.","PeriodicalId":43808,"journal":{"name":"International Journal of Engineering and Technology Innovation","volume":" ","pages":""},"PeriodicalIF":1.3000,"publicationDate":"2023-07-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"International Journal of Engineering and Technology Innovation","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.46604/ijeti.2023.11510","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"ENGINEERING, MULTIDISCIPLINARY","Score":null,"Total":0}
引用次数: 0

Abstract

Sentiment analysis (SA) has become an essential component of natural language processing (NLP) with numerous practical applications to understanding “what other people think”. Various techniques have been developed to tackle SA using deep learning (DL); however, current research lacks comprehensive strategies incorporating multiple-word embeddings. This study proposes a self-attention mechanism that leverages DL and involves the contextual integration of word embedding with a time-dispersed bidirectional gated recurrent unit (Bi-GRU). This work employs word embedding approaches GloVe, word2vec, and fastText to achieve better predictive capabilities. By integrating these techniques, the study aims to improve the classifier’s capability to precisely analyze and categorize sentiments in textual data from the domain of movies. The investigation seeks to enhance the classifier’s performance in NLP tasks by addressing the challenges of underfitting and overfitting in DL. To evaluate the model’s effectiveness, an openly available IMDb dataset was utilized, achieving a remarkable testing accuracy of 99.70%.
一种用于自注意机制的双向GRU结构:一种具有混合单词嵌入的自适应多层方法
情感分析(SA)已成为自然语言处理(NLP)的重要组成部分,在理解“别人的想法”方面有许多实际应用。已经开发了使用深度学习(DL)来解决SA的各种技术;然而,目前的研究缺乏将多个单词嵌入其中的综合策略。本研究提出了一种利用DL的自注意机制,并涉及单词嵌入与时间分散双向门控递归单元(Bi-GRU)的上下文集成。这项工作采用单词嵌入方法GloVe、word2vec和fastText来实现更好的预测能力。通过整合这些技术,本研究旨在提高分类器准确分析和分类电影领域文本数据中情感的能力。该研究试图通过解决DL中的欠拟合和过拟合问题来提高分类器在NLP任务中的性能。为了评估该模型的有效性,使用了一个公开可用的IMDb数据集,实现了99.70%的显著测试准确率。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
CiteScore
2.80
自引率
0.00%
发文量
18
审稿时长
12 weeks
期刊介绍: The IJETI journal focus on the field of engineering and technology Innovation. And it publishes original papers including but not limited to the following fields: Automation Engineering Civil Engineering Control Engineering Electric Engineering Electronic Engineering Green Technology Information Engineering Mechanical Engineering Material Engineering Mechatronics and Robotics Engineering Nanotechnology Optic Engineering Sport Science and Technology Innovation Management Other Engineering and Technology Related Topics.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信