Anatomical Planes-Based Representation for Recognizing Two-Person Interactions from Partially Observed Video Sequences: A Feasibility Study

Abstract

This paper presents a new approach for two-person interaction recognition from partially observed video sequences. The proposed approach employs the 3D joint positions captured by a Microsoft Kinect sensor to construct a view-invariant anatomical planes-based descriptor, called the two-person motion-pose geometric descriptor (TP-MPGD), that quantifies the activities performed by two interacting persons at each video frame. Using the TP-MPGDs extracted from the frames of the input videos, we construct a two-phase classification framework to recognize the class of the interaction performed by two persons. The performance of the proposed approach has been evaluated using a publicly available interaction dataset that comprises the 3D joint positions data recorded using the Kinect sensor for 21 pairs of subjects while performing eight interactions. Moreover, we have developed five different evaluation scenarios, including one evaluation scenario that is based on fully observed video sequences and four other evaluation scenarios that are based on partially observed video sequences. The classification accuracies obtained for each of the five evaluation scenarios demonstrate the feasibility of our proposed approach to recognize two-person interactions from fully observed and partially observed video sequences.