Recognizing and Localizing Individual Activities through Graph Matching

Abstract

In this paper we tackle the problem of detecting individualhuman actions in video sequences. While the most successfulmethods are based on local features, which proved thatthey can deal with changes in background, scale and illumination,most existing methods have two main shortcomings:first, they are mainly based on the individual power ofspatio-temporal interest points (STIP), and therefore ignorethe spatio-temporal relationships between them. Second,these methods mainly focus on direct classification techniquesto classify the human activities, as opposed to detectionand localization. In order to overcome these limitations,we propose a new approach, which is based on agraph matching algorithm for activity recognition. In contrastto most previous methods which classify entire videosequences, we design a video matching method from twosets of ST-points for human activity recognition. First,points are extracted, and a hyper graphs are constructedfrom them, i.e. graphs with edges involving more than 2nodes (3 in our case). The activity recognition problemis then transformed into a problem of finding instances ofmodel graphs in the scene graph. By matching local featuresinstead of classifying entire sequences, our methodis able to detect multiple different activities which occursimultaneously in a video sequence. Experiments on twostandard datasets demonstrate that our method is comparableto the existing techniques on classification, and that itcan, additionally, detect and localize activities.