A keyframe weighted dual-channel attention GCN model for human skeleton motion prediction

Abstract

Accurate prediction of human skeletal motion sequences is critical for human activity analysis and low-latency motion reconstruction applications. While many studies focus on frame-by-frame prediction model designs, the keyframes in a motion sequence may contain more spatial-temporal information than the other keyframes do. To address the importance of keyframes, this work introduces a heterogeneous keyframe selection and fusion method to discriminate the importance of different motion frames from historical observations for prediction. Specifically, we propose an adaptive keyframe selection algorithm to iteratively select the keyframes and a nonlinear heterogeneous interpolation method to reconstruct the transitional frames. By merging them with the original motion sequence, the semantics of the original motion are preserved, and the importance of the keyframes is highlighted. A graph convolutional network (GCN) is designed for prediction with dual-channel attention to incorporate motion patterns in longer-term historical records to improve motion feature exploration. A comprehensive evaluation of the model is performed on the Human3.6M and AMASS datasets, which shows significant improvement in motion prediction over long-term methods (\(\ge \) 320 ms) over the state-of-the-art methods in terms of the 3D mean per joint position error (MPJPE).