Construction of word level Tibetan Lip Reading Dataset

Abstract

With the development of deep learning technology, dataset always play an important role in different research fields. Lip reading, which involves image processing and natural language processing, has become one of the most challenging research topics in the field of deep learning. However, the diversity of lip changes and the richness of language itself greatly improve the difficulty of lip reading, which leads to the slow progress of lip reading research. In order to provide a good basis for future Tibetan lip reading, this paper constructs the first Tibetan lip reading dataset, named TLRW-50, which is saved as a series of lip-shaped image sequences after data preprocessing. The complete process and algorithm details of the Tibetan lip reading dataset are proposed and the quality of the lip reading video is evaluated. Six methods of image data expansion are used to expand the lip image frame sequence: color enhancement, Gaussian noise, horizontal image, amplification, rotation and clipping. Ten Tibetan speakers were selected to evaluate the quality of the cut lip reading video before the expansion of the lip image sequence. The MOS score was 3.8.