An Lipreading Modle with DenseNet and E3D-LSTM

Abstract

For Chinese lip reading, the paper proposes an improved DenseNet network structure to strengthen the ability of short-term dependence of the model. At the backend,E3D-LSTM is adopted to conduct time modeling of features extracted from CNN. The loss function of CTC is used to solve different speech habits of speakers, which will lead to different time dependence of the same word. LRW-1000 datasets are used for training, and experiments show that the Chinese recognition rate is better than the traditional method. In the diffferent length of sequences, our work arrives 38.96%in easy,38.49% in medium and 37.92% in hard, respectively.