Japanese Sentence Dataset for Lip- reading

Abstract

This research is about lip-reading for Japanese sentences. Research on English sentences is actively pursued due to the extensive datasets. However, a sufficient dataset for Japanese sentences has not been released. Therefore, this paper builds a Japanese sentence dataset. A Transformer model is used for the recognition task. Three recognition target levels: phoneme, mora, and vowel, are set, and recognition experiments show that they can be recognized.