Spatiotemporal auto-correlation of grayscale gradient with importance map for cooking gesture recognition

Abstract

We propose a gesture recognition method employing spatiotemporal auto-correlation of grayscale gradient for image sequences capturing cooking activities. Recognizing gestures in housework activities is a key technology for realizing sophisticated household devices, energy saving as well as supporting elder or handicapped people. The proposed method employs Cubic Gradient Local Auto Correlation (Cubic GLAC) to describe shape of objects and its temporal change in a video sequence. Human gestures are able to be recognized by not only appearance and motion but environmental objects. Actually, cooking gestures also have strong relationship to surrounding kitchen utensils. To utilize this observation for gesture recognition, we introduce the importance map that restricts regions of interest for recognition. Support vector machine with linear kernel is employed to classify the extracted feature among 10 gesture classes. Performance evaluation experiment using "Actions for Cooking Eggs (ACE)" Dataset, which is an open dataset for context-based gesture recognition, shows that the proposed method outperforms recognition methods using similar spatiotemporal features.