Actor Spatiotemporal Relation Networks for Group Activity Recognition

The existing video-based group activity recognition methods do not full use spatiotemporal information, and cannot effectively improve the accuracy of group recognition. This paper proposes an effective group activity recognition model based on Actor Spatial-Temporal Relation Networks (ASRN) to capture potential spatiotemporal features in an end-to-end manner. First, we propose an SRM to get the feature correlation between feature nodes from the temporal dimension and the spatial dimension. Second, Personal Spatiotemporal Feature Module (PSFM) and a Multi-actors Relation Module (MRM) are designed using SRM to extract actor-level spatiotemporal semantic information and the relation features between actors. We conducted experiments on two datasets: volleyball dataset and collective activity dataset. The results on these two datasets show the superiority of our method.