Greek sign language vocabulary recognition using Kinect

Abstract

Sign language recognition is a challenging problem both when tracking continuous signs (communication mode) or single words (translation mode)1. We have developed a system that can recognize Greek sign language vocabulary in translation mode using Kinect technology. The sensor captures 3D hands movement trajectory and then a set of features in the form of body joints are fed to a classifier to recognize the input sign. Normalization is used to align test and stored trajectories using the dynamic time warping algorithm before matching is done using the Nearest-Neighbor approach. The low computational complexity of the involved algorithms allows for building a system with real-time response times. The system was evaluated with a sample of 5 individuals and is capable of recognizing 15 signs of the Greek sign language. Different configurations were tested and the best accuracy achieved was 99.33%.