Japanese Fingerspelling Recognition Based on Classification Tree and Machine Learning

Abstract

Sign language is a very important communication tool for hearing-impaired people and also for the communication between hearing-impaired and non-handicapped people. There are many methods for sign language recognition, some of which are based on Hidden Markov Model (HMM) and others are based on Support Vector Machine (SVM) and so forth. In fact, the most of previous methods recognize fingerspelling using video sequence because some fingerspellings are accompanied by movement. Some methods use Microsoft Kinect or Leap Motion controller to obtain the finger movement. Some fingerspellings, however, are not accompanied by movement and can be recognized with just one snap shot of fingerspelling. Therefore, this paper proposes a recognition method of fingerspelling without movement. The target fingerspellings are 41 characters without movement in Japanese sign language, and the method uses only one picture. Some of fingerspellings are easily recognized and others are not so that the method is based on pattern recognition using classification tree and machine learning with SVM for easily recognized fingerspellings and difficultly recognized ones, respectively. As the result of the experiment, the averaged recognition rate was 86%.