Multi-3D pose tracking based on multi-view fusion feature correlation

Abstract

Pedestrian 3D pose tracking in multi-view scenarios has extensive practical applications. However, existing methods often overlook the overall tracking accuracy of pedestrians, particularly the issues of missing and erroneous tracking caused by severe occlusions, disappearances, and reappearances. It further affects the accuracy of pose point association. To address these limitations, a two-stage method is proposed, involving tracking with an exceptionally low error rate, followed by obtaining higher precision 3D pose points. Firstly, a multi-object tracking model is introduced, which integrates feature association-validation-updating and employs dynamic thresholding strategy to achieve high-accuracy matching of multiple individuals in multi-view scenarios by computing similarity with feature pool templates. Additionally, a Gaussian Mixture-based feature pool updating model ensures the universality of stored features to solve the reappearance problem. Secondly, a pedestrian 2D pose detection and 3D pose reprojection method based on SMPL (Skinned Multi-Person Linear model) is proposed, which detects more complete pose points than OpenPose in complex scenes and better conforms to the distribution principles of human skeletal pose points. To validate the advancedness of the proposed method, the Shelf and Campus public datasets are re-annotated. Experimental results demonstrate the excellent performance of the proposed method in overall error control in complex environments, outperforming existing methods in multi-object tracking and pose point estimation accuracy and completeness.