Spatiotemporal saliency and sub action segmentation for human action recognition

Abstract

Human Action Recognition is a significant and challenging field of interest in Research and Industry. In this paper, the Selective Spatiotemporal Interest Points (Selective STIPs) are extracted from the input video and is labeled using a dictionary. The actions are segmented into sub-actions, and then the temporal and spatial structure is captured. The segmentation is done on the basis of interest point density. The spatial and temporal relationships between the labeled STIPs is represented using Space Salient and Time Salient directed graphs respectively. Time Salient pairwise feature (TSP) and Space Salient pairwise feature (SSP) is computed from corresponding directed graphs. The Selective STIP suppresses the background STIPs and detects more robust STIPs from the actors which improves performance of recognition. The Bag-of-Visual Words model combined with TSP and SSP for human action classification provides a more promising result.