Time-weighted motion history image for human activity classification in sports

Abstract Vision-based human activity classification has remarkable potential for various applications in the sports context (e.g., motion analysis for performance enhancement, active sensing for athletes, etc.). Recently, learning-based human activity classifications have been widely researched. However, in sports scenes in which more detailed and player-specific classifications are required, this is a quite challenging task; in many cases, only a limited number of datasets are available, unlike daily movements such as walking or climbing stairs. Therefore, this paper proposes a time-weighted motion history image, an effective image sequence representation for learning-based human activity classification. Unlike conventional MHI based on the assumption that “the newer frame is more important,” our method generates importance-aware representation so that the predictor can “see” the frames that contribute to analyzing the specific human activity. Experimental results have shown the superiority of our method.