Cooking gesture recognition using local feature and depth image

In this paper, we propose a method combining visual local features and depth image information to recognize cooking gestures. We employ the feature calculation method[2] which used extended FAST detector and a compact descriptor CHOG3D to calculate visual local features. We pack the local features by BoW in frame sequences to represent the cooking gestures. In addition, the depth images of hands gestures are extracted and integrated spatio-temporally to represent the position and trajectory information of cooking gestures. The two kinds of features are used to describe cooking gestures, and recognition is realized by employing the SVM. In our method, we determine the gesture class for each frame in cooking sequences. By analyzing the results of frames, we recognize cooking gestures in a continue frame sequences of cooking menus, and find the temporal positions of the recognized gestures.