Sign Language Recognition Using Convexity Defects and Hough Line

This paper presents a novel feature extraction framework for sign language recognition application to assist the deaf and speech-impaired in daily communication. Our method extracts visual features from the hand gesture using convexity defects, K-curvature and Hough line which can differentiate visually similar signs. In segmentation stage, Canny edge detection and skin color segmentation with histogram backprojection method are used to extract the hand region from the background. The features to be extracted are categorized into shape, orientation and motion. The shape features of the sign language will then be extracted using convexity defect, K-curvature and Hough line techniques. The orientation feature will be extracted by calculating the palm center to wrist angle. The trajectory motion of the dynamic gesture is extracted using the chain code method. Lastly, Decision Tree classification is employed in classification of both static and dynamic gestures. The proposed framework is carried out in smartphone platform to recognize 26 alphabets, 10 numbers and eight dynamic American Sign Language (ASL). The average accuracy of 29 static ASL achieved is 85.72% and average accuracy of 10 dynamic ASL achieved is 77%. Through this research, it is found that the proposed framework can recognize larger sign languages database as compared with previous convexity defect-based sign language recognition research. *Corresponding Author: Ming Jin Cheok, Universiti Teknologi Malaysia, Skudai, 81300, Malaysia; E-mail: mingjin_91@hotmail.com Citation: Cheok MJ, Omar Z, Jaward MH (2020) Sign Language Recognition Using Convexity Defects and Hough Line. Int J Comput Softw Eng 5: 158. doi: https:// doi.org/10.15344/2456-4451/2020/158 Copyright: © 2020 Cheok et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Some notable appearance-based feature extraction method includes the extraction of Shift-Invariant Feature Transform (SIFT) features from the gesture. For example, in paper [13], SIFT features are extracted from six signs and the features are rotational invariant. Research in [14,15] also extracted SIFT features from the hand gestures, and the features are simplified by first quantizing with K-means clustering and then mapped into Bag-of-Features (BoF). Representing SIFT features using BoF reduces and uniforms the dimensionality of each SIFT features extracted. Speeded Up Robust Feature (SURF) is another notable feature extraction method proposed in [16]. The authors in paper [17] extracted SURF features from a moving hand gestures of consecutive frames, and by analyzing correlation between SURF points, classification of 26 gestures achieves 84.6% accuracy. Computational performance of SIFT and SURF has been compared in paper [15], as the extraction of features is usually a computationally heavy process and hence the efficiency of the technique plays an important role. It is showed that SURF has a faster processing speed. Principle Component Analysis (PCA) is another commonly extracted features in hand gesture recognition research. In paper [18], PCA is extracted together with kurtosis position and chain code, where kurtosis position is used in finding edges and chain code is used in tracking the hand trajectory. The hybrid of these three features has shown to outperform any features alone with an error rate of 10.91%. Model-based sign language recognition based on convexity defect features has been widely applied in research in this field [19-30]. The advantage of using this approach as compared to other appearanceInternational Journal of Computer & Software Engineering Ming Jin Cheok1,*, Zaid Omar1 and Mohamed H. Jaward2 1Universiti Teknologi Malaysia, Skudai, 81300, Malaysia 2School of Engineering, Monash University Subang Jaya, 47500, Malaysia Int J Comput Softw Eng IJCSE, an open access journal ISSN: 2456-4451 Volume 5. 2020. 158 Cheok et al,. Int J Comput Softw Eng 2020, 5: 158 https://doi.org/10.15344/2456-4451/2020/158