Sign recognition using depth image streams

Abstract

A set of techniques is presented for extracting essential shape information from image sequences. Presented methods are (i) human detection, (ii) human body parts detection, and (iii) hand shape analysis, all based on depth image streams. In particular, representative types of hand shapes used in Japanese sign language (JSL) are recognized in a non-intrusive manner with a high recognition rate. An experimental JSL recognition system is built that can recognize over 100 words by using an active sensing hardware to capture a stream of depth images at a video rate. Experimental results are shown to validate our approach and characteristics of our approach are discussed