Korean sign language recognition based on image and convolution neural network

Abstract

The purpose of this paper is to develop a convolution neural network based model for Korean sign language recognition. For this purpose, sign language videos were collected for 10 selected words of Korean sign language and these videos were converted into images to have 9 frames. The images with 9 frames were used as input data for the convolution neural network based model developed in this study. In order to develop the model for Korean sign language recognition, experiments for determining the number of convolution layers was first performed. Second, experiments for the pooling which intentionally reduces the features of the feature map was performed. Third, we conducted an experiment to reduce over fitting in the model learning process. Based on the experiments, we have developed a convolution neural network based model for Korean sign language recognition. The accuracy of the developed model was about 84.5% for the 10 selected Korean sign words.