Spatial–Temporal–Geometric Graph Convolutional Network for 3-D Human Pose Estimation From Multiview Video

Abstract

The multiview 3-D human pose estimation (HPE) effectively addresses challenges, such as depth ambiguity and occlusion faced by monocular methods through the complementing of geometric information from multiple views. However, existing multiview methods often necessitate well-calibrated camera parameters or rely on complex parametric models. These requirements can result in inaccuracies when camera placement is perturbed and can negatively impact the deployability. This article proposes a lightweight approach that synergistically models geometric information with spatial-temporal information without relying on camera parameters, named spatial-temporal–geometric graph convolutional network (STG-GCN). We leverage the inherent connections in multiview sequences of 2-D poses, representing them as a spatial-temporal–geometric graph (STG-Graph), which allows for the simultaneous encoding of spatial-temporal–geometric relations across various joints, consecutive frames, and multiple views. Using a unified graph to model all features, this approach reduces the parameter explosion in existing methods, caused by separate modules extracting spatial, temporal, and view axis features. Building upon the STG-Graph, an adaptive confidence-aware graph convolution (ACA-GraphConv) is proposed to mitigate the impact of unreliable 2-D poses predicted by 2-D pose estimators. This is achieved by leveraging corresponding confidence scores to adjust the graph convolution accordingly. Experimental results on two public datasets demonstrate that our STG-GCN achieves performance comparable to state-of-the-art approaches while significantly reducing parameter volume. Ablation studies also illustrate the effectiveness of our ACA-GraphConv in both monocular and multiview scenarios.