Chinese Finger Sign Language Recognition Method with ResNet Transfer Learning

Abstract

Sign language is one of the most effective ways to help hearing-impaired people to communicate with other people. Although deep learning methods have been used in recognition, there are still problems with finger sign language recognition. The major issue is that the gradient approach usually fails, or the obtained recognition accuracy is not high when the depth is increasing. We thus propose a Chinese finger sign language recognition method based on ResNet and Adam optimizer together with additional image processing techniques to gain higher accuracy. We then compare our recognition results to other convolutional neural network models which are widely used deep learning techniques for recognition. Even though we have a small size of the dataset, our proposed deep learning method for finger sign recognition still gives a higher recognition rate. Also, our prototypical method provides the capability to be applied to other recognition tasks of different gestures or objects from image datasets.