Action Recognition with Trajectory and Scene

Abstract

Trajectory features have recently shown promising results to action recognition in video. Typically, they are extracted by tracking feature points with the KLT tracker or matching SIFT descriptors between frames. However, trajectory can be due to the action of interest, but also be caused by background or the camera motion. To overcome the problem, human detection is applied to roughly estimate of the location of the human in the video and segment video into Foreground/Background regions. In many cases, human actions can be identified not only by observing human body in motion, but also properties of the surrounding scene. In our work, we addresse the problem and propose an approach that integrates multiple features from scene and people. We evaluate our video description with a bag of-features model. We also present experimental results on two datasets with an increasing degree of difficulty and demonstrate significant improvements.