Visible speech modelling and hybrid hidden Markov models/neural networks based learning for lipreading

A. Rogozan, P. Deléglise
{"title":"Visible speech modelling and hybrid hidden Markov models/neural networks based learning for lipreading","authors":"A. Rogozan, P. Deléglise","doi":"10.1109/IJSIS.1998.685470","DOIUrl":null,"url":null,"abstract":"This paper describes a new approach for automatic visible speech recognition based on hybrid hidden Markov models/neural networks. Suitable geometric features extracted from speaker's lip shapes are used to train the speech recognizer with nonsense sentences. First we describe the use of a geometrical-based model for visible speech and we outline a self-organising-map-based approach to determine the visual specific recognition units suitable for our speaker-dependent visible speech recognition task. Then we describe five automatic lipreading systems we developed according to different classification techniques: hidden Markov models, neural networks and hybrid hidden Markov models/neural networks. All these systems are tested on a connected letter recognition task. Finally, the performance comparison underlines that a hybrid hidden Markov models/neural networks based architecture is the most promising for automatic lipreading purposes.","PeriodicalId":289764,"journal":{"name":"Proceedings. IEEE International Joint Symposia on Intelligence and Systems (Cat. No.98EX174)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"1998-03-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"4","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Proceedings. IEEE International Joint Symposia on Intelligence and Systems (Cat. No.98EX174)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/IJSIS.1998.685470","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 4

Abstract

This paper describes a new approach for automatic visible speech recognition based on hybrid hidden Markov models/neural networks. Suitable geometric features extracted from speaker's lip shapes are used to train the speech recognizer with nonsense sentences. First we describe the use of a geometrical-based model for visible speech and we outline a self-organising-map-based approach to determine the visual specific recognition units suitable for our speaker-dependent visible speech recognition task. Then we describe five automatic lipreading systems we developed according to different classification techniques: hidden Markov models, neural networks and hybrid hidden Markov models/neural networks. All these systems are tested on a connected letter recognition task. Finally, the performance comparison underlines that a hybrid hidden Markov models/neural networks based architecture is the most promising for automatic lipreading purposes.
基于唇读的可见语音建模和混合隐马尔可夫模型/神经网络学习
本文提出了一种基于隐马尔可夫模型和神经网络的自动可视语音识别方法。从说话人的唇形中提取合适的几何特征来训练无意义句子的语音识别器。首先,我们描述了基于几何的可见语音模型的使用,并概述了一种基于自组织映射的方法,以确定适合于我们的依赖于说话人的可见语音识别任务的视觉特定识别单元。然后描述了我们根据不同的分类技术开发的五种自动唇读系统:隐马尔可夫模型、神经网络和混合隐马尔可夫模型/神经网络。所有这些系统都在一个连接的字母识别任务上进行了测试。最后,性能比较强调了基于隐马尔可夫模型/神经网络的混合架构对于自动唇读目的最有前途。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信