Inflated 3D Architecture for South Indian Sign Language Recognition

Abstract

The inability to speak is considered to be a true disability that affects both the speaker and the listener. Therefore, there is a need for a method that can recognize sign languages so that a successful communication is established. This paper proposes a method that classifies South-Indian Sign Language and translates it in real-time to Kannada language. The first few samples for the dataset were obtained from the official ISL website while the rest of the samples were manually recorded. Optical flow method is employed to extract the motion features in a video and the resulting frames are used as input to the proposed Inflated-3D model which recognizes the sign language. Stratified k-fold cross- validation is used to improve the performance. The predicted sign in text form is then translated to Kannada language through the use of GoogleTrans API and further synthesized into a speech segment using the GTTS open-source library. In addition, a comparative study of the proposed methodology with other techniques that have been proposed to recognize sign languages through video streams has been presented. The proposed method resulted in an average accuracy of 0.8709.