{"title":"Visible speech modelling and hybrid hidden Markov models/neural networks based learning for lipreading","authors":"A. Rogozan, P. Deléglise","doi":"10.1109/IJSIS.1998.685470","DOIUrl":null,"url":null,"abstract":"This paper describes a new approach for automatic visible speech recognition based on hybrid hidden Markov models/neural networks. Suitable geometric features extracted from speaker's lip shapes are used to train the speech recognizer with nonsense sentences. First we describe the use of a geometrical-based model for visible speech and we outline a self-organising-map-based approach to determine the visual specific recognition units suitable for our speaker-dependent visible speech recognition task. Then we describe five automatic lipreading systems we developed according to different classification techniques: hidden Markov models, neural networks and hybrid hidden Markov models/neural networks. All these systems are tested on a connected letter recognition task. Finally, the performance comparison underlines that a hybrid hidden Markov models/neural networks based architecture is the most promising for automatic lipreading purposes.","PeriodicalId":289764,"journal":{"name":"Proceedings. IEEE International Joint Symposia on Intelligence and Systems (Cat. No.98EX174)","volume":"1 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"1998-03-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"4","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Proceedings. IEEE International Joint Symposia on Intelligence and Systems (Cat. No.98EX174)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/IJSIS.1998.685470","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 4
Abstract
This paper describes a new approach for automatic visible speech recognition based on hybrid hidden Markov models/neural networks. Suitable geometric features extracted from speaker's lip shapes are used to train the speech recognizer with nonsense sentences. First we describe the use of a geometrical-based model for visible speech and we outline a self-organising-map-based approach to determine the visual specific recognition units suitable for our speaker-dependent visible speech recognition task. Then we describe five automatic lipreading systems we developed according to different classification techniques: hidden Markov models, neural networks and hybrid hidden Markov models/neural networks. All these systems are tested on a connected letter recognition task. Finally, the performance comparison underlines that a hybrid hidden Markov models/neural networks based architecture is the most promising for automatic lipreading purposes.