Thai Sign Language Recognition Using 3D Convolutional Neural Networks

Abstract

Translating sign language is a complicating and challenging task due to complexity of sign language structures including hand shapes, hand orientation, hand movements, and facial expression. Existing approaches generally used complex handcrafted features to recognize a sign language. However, to build a model based on those features is a difficult task. To deal with this problem, we proposed a model of Thai sign language recognition using 3D Convolutional Neural Network (3DCNN) which can automatically learn both temporal and spatial features from data. Our data is a collection of 64 isolated Thai signs language vocabulary as video stream using Microsoft Kinect to acquire information of color, depth, skeleton, hand shapes, and whole body movement. To evaluate our proposed approach, different kind of stream and information are tested with the best empirical model. The selected model is 3DCNN with kernel 3×3×3. The experimental results demonstrated that the accuracy of different input and information is as high as the highest 97.7% which belong to skeleton dataset.