Multi-Scale Feature Fusion Network for Lip Recognition

Abstract

Visual speech recognition (VSR) is also known as lip recognition. Recently, it has been widely explored due to the development of deep learning. Lip recognition is a discrimination issue, where the information provided by the delicate movement of the lips is most remarkable of all. This places a higher demand on the model's ability to extract features of minor variation around the lips. In this paper, a three-dimensional convolutional network (3D CNN) multi-branch feature fusion network is proposed for extracting spatiotemporal featuresof continuous images. The features of multi-branch feature fusion network are utilized to fully extract partial and general characteristics from sequential imagery and further enhance the feature information to deliver more accurate function info to the back-end classification network. The excellence of quite a few methods requires the support of huge volume of data, and in favor of test the effect of small-scale data sets. This experimentis conducted using the Oulu Vs2dataset to obtain exciting experimental results. After 20 iterations of the experiment, the maximum accuracy absolutely improves by 0.8% and the average accuracy improves by 1%.