Smart Tutoring System for Arabic Sign Language Using Leap Motion Controller

Abstract

This paper presents a smart tutoring system for Arabic Sign Language (ArSL). Sign language is one of the main approaches of communication for people with hearing impairment. Many people are willing to learn sign language and support this segment of the society; however, learning this language requires some effort and assistant. Tools that are used to support sign language learners and specifically ArSL are limited and insufficient. Hence, the development of a tool that is capable of training and assessing ArSL learners becomes a necessity. We proposed a smart tutoring for ArSL based on using the leap motion’s hand tracking technology. The aim of this system is assisting non-disabled learners who want to learn the sign language, such as undergraduates specializing in hearing disabilities, parents of kids with hearing impairment or any interested subject. The system allows learners to practice ArSL in different levels and self-assess themselves. As it utilizes the recent technology of leap motion controller, it can detect and track hand and fingers movements and consequently assess the position and movement accuracy. Machine learning techniques, specifically the K- Nearest Neighbor algorithm was applied for classification and sign recognition. Preliminary prototype was developed and tested in terms of users’ acceptance. The outcomes show satisfactory and promising results. It is expected that the proposed system will contribute in enriching the learning process of ArSL and consequently support an important segment of our community.