Spatio-Temporal Feature Extraction and Distance Metric Learning for Unconstrained Action Recognition

Abstract

In this work, we proposed a framework for zero-shot action recognition with spatio-temporal feature (ST-features) in order to address the problem of unconstrained action recognition. It is more challenging than the constrained action recognition problem, since a model has to recognize actions which do not appear in the training step. The proposed framework consists of two models: 1) ST-feature extraction model and 2) verification model. The ST-feature extraction model extracts discriminative ST-features from a given video clip. With these features, the verification model computes the similarity between them to examine class-identity whether their classes are identical or not. The experimental results show that the proposed framework can outperform other action recognition methods under the unconstrained condition.