Utilization of stereo disparity and optical flow information for human interaction

Abstract

To attain smooth human interaction we propose a system which simultaneously utilizes the stereo disparity and optical flow information of real-time stereo gray and multi resolution images to recognize objects and gestures. For real-time calculation of the disparity and optical flow information of a stereo image the system first creates pyramid images by utilizing a Gaussian filter. The system then determines the disparity and optical flow of a low density image and extracts regions in front of a certain depth. The three foremost regions are recognized by higher order local autocorrelation features and a linear discriminant analysis. With this process the system recognizes the face and hand signs of users which are displayed foremost, and roughly recognizes movements within the region in real-time. With this framework, the system can discriminate, the face of a user, can monitor the basic movements of the user, can smoothly learn a presented object by users and can communicate with users from hand signs learned in advance.