ESMAANI: A Static and Dynamic Arabic Sign Language Recognition System Based on Machine and Deep Learning Models

Abstract

As the size of the population of sign language users increased, the importance of breaking the barrier between those who can use sign language and those who can not in the Arabic community increased. In this paper, We present ESMAANI, a computational solution that enables sign language recognition while utilizing machine learning and deep learning techniques. The proposed system aims to contribute to the study of the challenges and complexities associated with sign language recognition, specifically Arabic sign language. The proposed models present a non-intrusive computer vision approach to building a system specialized in Arabic sign language recognition translating the input sign gestures from a camera stream or video input into text output. Supporting static sign language input, which is common in fingerspelling and alphabet representation and dynamic sign language input which is employed for signing at the word level. The paper also presents a person and environment-independent dataset that's capable of generalizing to include further the various versions of ArSL the proposed static sign recognition system achieved an overall accuracy of 99.7%. And For the proposed dynamic sign recognition system achieved maximum recognition validation accuracy of 97% suggesting strong generalization.