A Proposal for Creating Syllabic Datasets for Japanese Language Lipreading by Using Machine Learning

Abstract

Although lip-reading using image processing and machine learning has been mainly performed at the word level, it has been shown that using LipNet, a network that enables recognition at the sentence level, improves the recognition accuracy over the former method. However, this was the case for English speakers. In this study, the data set was created based on speech scenes containing all 50 Japanese sounds, and the recognition accuracy was evaluated using LipNet.