Developing Automatic Markerless Sign Language Gesture Tracking and Recognition System

Abstract

Machine-based interpretations of sign language hand postures and gestures have long been a specialized research topic in human-computer interaction. Involvement of human subjects and multi-faceted nature of the problem requiring expertise in a multitude of disciplines make sign language interpretation an exigent research problem. Purpose of this paper is to present the experience and findings of international collaborative research conducted over several years on vision-based sign language translation. At first, this paper extends a discussion on sign language and its variations, machine-based translation and its significance. Secondly, a discussion on how three main tasks within the machine translation, namely: a) hand localization and tracking, b) hand posture interpretation, and c) hand gesture interpretation, can be addressed. Finally, research challenges, possible approaches, and future extensions are discussed.