Relevant features for video-based continuous sign language recognition

Abstract

This paper describes the development of a video-based continuous sign language recognition system. The system is based on continuous density hidden Markov models (HMM) with one model for each sign. Feature vectors reflecting manual sign parameters serve as input for training and recognition. To reduce computational complexity during the recognition task beam search is employed. The system aims for an automatic signer-dependent recognition of sign language sentences, based on a lexicon of 97 signs of German sign language (GSL). A further colour video camera is used for image recording. Furthermore the influence of different features reflecting different manual sign parameters on the recognition results are examined. Results are given for varying sized vocabulary. The system achieves an accuracy of 91.7% based on a lexicon of 97 signs.