HFNet: A Novel Model for Human Focused Sports Action Recognition

Abstract

Action recognition has attracted much attention recently and progressed remarkably. However, as a special kind of actions, sports action recognition is more difficult and deserves more attention. Our goal in this paper is to distinguish fine-grained human-focused sport actions. Sport actions can always be decomposed into sub-actions by body parts and it's necessary to establish the relationships among body parts and combine them together to perform classification. Besides, sport actions are usually fine-grained and subclasses a re similar which are hard to distinguish. Another tough problem in practice is to locate the actor in complicated circumstances. However, current methods in action recognition always pay attention to the whole image, thus failing to capture details and constructing relationships in images. In this paper, we propose a novel model to construct visual relationships in images through graph convolutions. We make use of patches cropped around body joints as input for graph nodes. Thus our model is able to pay attention to the changes and details of body parts. Then, we carefully design model to learn connections among graph nodes adaptively and empirically. We also provide another method to construct visual relationships for graph nodes. By specially focusing on relationships and details, our model achieves start-of-the-art performance on complex human-focused sports datasets FSD-10 and Diving48.