Enhancing video-based human mesh recovery with Attention-Mamba synergy

While significant progress has been made in recovering 3D human body mesh from a single image, the domain of 3D human motion recovery from video still offers ample room for improvement in accuracy and smoothness. Existing video-based approaches typically employ singular methods to extract spatiotemporal features from video data for estimating complex pose and shape parameters to reconstruct human body mesh. However, their generalized feature extraction methods and the limited representational capacity of parametric models often lead to discontinuous motion poses and inaccurate body shapes. To solve these issues, we decouple video data temporal and spatial features leveraging Mamba (a kind of state-space sequence model) and Attention Mechanism (a learnable model focusing on the relevance features of the input). Additionally, we propose a Multihead Adaptive Agent Attention (a new type of attention mechanism, referred to as MAAA) module, which is applied to the non-parametric mesh reconstruction method to improve the mesh recovery effect. The proposed method consists of two stages: (1) 3D human pose estimation from videos and (2) reconstruction of human body mesh based on 3D pose and RGB information from videos. Quantitative and qualitative experiments demonstrate that our approach surpasses previous methods in temporal continuity and spatial accuracy on public datasets, such as enhancing 4.4% in MPVPE and 9.7% in ACCEL on Human3.6M dataset.