A lightweight Future Skeleton Generation Network(FSGN) based on spatio-temporal encoding and decoding

Abstract

Since early warning in industrial applications is far more valuable than post-event analysis, human activity prediction based on partially observed skeleton sequences has become a popular research area. Recent studies focus on building complex deep learning networks to generate accurate future skeleton data, but overlook the requirement for timeliness. Different from such frame-by-frame generation methods, we propose a Future Skeleton Generation Network (FSGN) based on spatio-temporal encoding and decoding framework. Firstly, we design a dynamically regulated input module to ensure equal-length input of partially observed data, and set modules like discrete cosine transform(DCT) and low-pass filtering(LPF) to filter important information. Then, we employ an improved multi-layer perceptron(MLP) structure as the basic computational unit for the encoding and decoding framework to extract spatio-temporal information, and propose using multi-dimensional motion error of human skeleton to form the loss function. Finally, we use an output module symmetrical to the input module to achieve the generation of future activity data. Results show that the proposed FSGN achieves fewer parameters(0.12 M) and higher generation accuracy, which can effectively provide future information for human activity prediction tasks.