Information Distance Based Self-Attention-BGRU Layer for End-to-End Speech Recognition

Yunhao Yan, Qinmengying Yan, Guang Hua, Haijian Zhang
{"title":"Information Distance Based Self-Attention-BGRU Layer for End-to-End Speech Recognition","authors":"Yunhao Yan, Qinmengying Yan, Guang Hua, Haijian Zhang","doi":"10.1109/ICDSP.2018.8631855","DOIUrl":null,"url":null,"abstract":"The common utilization of bidirectional gated recurrent unit (BGRU) architectures for end-to-end speech recognition suffers from long-term dependence and information redundancy. The reason lies in that the BGRU architectures model speech data according to time distance, which implicitly assumes that speech data is continuous. In this paper, we propose a new hypothesis, i.e., speech data possess the feature of being locally continuous and globally discrete. Based on this hypothesis, we propose to model speech data according to information distance. To support this hypothesis, we design an information distance based modeling architecture. Via the incorporation of self-attention mechanism, the proposed architecture is termed self-attention bidirectional gated recurrent unit (SABGRU). Experiment results show that SABGRU increases more than 10% speech recognition accuracy over conventional BGRU.","PeriodicalId":218806,"journal":{"name":"2018 IEEE 23rd International Conference on Digital Signal Processing (DSP)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2018-11-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"2018 IEEE 23rd International Conference on Digital Signal Processing (DSP)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/ICDSP.2018.8631855","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

Abstract

The common utilization of bidirectional gated recurrent unit (BGRU) architectures for end-to-end speech recognition suffers from long-term dependence and information redundancy. The reason lies in that the BGRU architectures model speech data according to time distance, which implicitly assumes that speech data is continuous. In this paper, we propose a new hypothesis, i.e., speech data possess the feature of being locally continuous and globally discrete. Based on this hypothesis, we propose to model speech data according to information distance. To support this hypothesis, we design an information distance based modeling architecture. Via the incorporation of self-attention mechanism, the proposed architecture is termed self-attention bidirectional gated recurrent unit (SABGRU). Experiment results show that SABGRU increases more than 10% speech recognition accuracy over conventional BGRU.
基于信息距离的自注意-端到端语音识别的bgru层
双向门控循环单元(BGRU)体系结构在端到端语音识别中的常用应用存在长期依赖和信息冗余的问题。原因在于BGRU架构根据时间距离对语音数据进行建模,隐含地假设语音数据是连续的。本文提出了一个新的假设,即语音数据具有局部连续和全局离散的特征。基于这一假设,我们提出了基于信息距离的语音数据建模。为了支持这一假设,我们设计了一个基于信息距离的建模体系结构。通过引入自注意机制,提出了自注意双向门控循环单元(SABGRU)。实验结果表明,与传统的BGRU相比,SABGRU的语音识别准确率提高了10%以上。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信