Leveraging edge detection techniques to enhance Arabic sign language static-gesture recognition using deep learning

Abstract

Recently, there has been a growing interest in developing solutions to address communication barriers for the deaf and hard-of-hearing community. Sign language is the primary language of this group. Computer vision technology is used to process sign language due to its ease of application. Sign language recognition involves the use of technology to bridge communication gaps and enhance accessibility for individuals who use sign language as their primary form of communication. Many researchers have presented various methods to facilitate communication, among others. These methods include sign language recognition techniques, translation between sign and text or audio, and hand gesture identification, among others. We proposed an effective approach to improve the feature extraction process for Arabic sign gesture recognition. Feature extraction is a crucial aspect of deep learning models because it facilitates data processing, improves performance, and helps interpret results. This process also enables models to manage large datasets more efficiently. We presented two deep learning models: the agile convolutional neural network (ASLR_CNN) and ResNet50, to improve the comprehensiveness of the extracted features. These models were combined with the Canny Edge Detector (CED), which identifies the edges of Arabic hand gestures, as well as the complex features extracted from the edges by the proposed models. To evaluate the effectiveness of our methodology, we trained the proposed models on two public datasets: AASL and ArASL. The performance of these models was evaluated using a variety of metrics, including accuracy, precision, recall, F-score, and confusion matrix. The results indicated that both the ASLR_CNN and ResNet50 models achieved high accuracy on the ArASL dataset, reaching 97.14 % and 96.88 %, respectively. In contrast, the accuracy dropped to 89.49 % and 86.12 % for the ASLR_CNN and ResNet50 models, respectively, when using the AASL dataset.