A Multi-Type Feature Fusion Network Based on Importance Weighting for Occluded Human Pose Estimation

Abstract

Human pose estimation is a challenging task in computer vision. Most algorithms perform well in regular scenes, but lack good performance in occlusion scenarios. Therefore, we propose a multi-type feature fusion network based on importance weighting, which consists of three modules. In the first module, we propose a multi-resolution backbone with two feature enhancement sub-modules, which can extract features from different scales and enhance the feature expression ability. In the second module, we enhance the expressiveness of keypoint features by suppressing obstacle features and compensating for the unique and shared attributes of keypoints and topology. In the third module, we perform importance weighting on the adjacency matrix to enable it to describe the correlation among nodes, thereby improving the feature extraction ability. We conduct comparative experiments on the keypoint detection datasets of common objects in Context 2017 (COCO2017), COCO-Whole-body and CrowdPose, achieving the accuracy of 78.9%, 67.1% and 77.6%, respectively. Additionally, a series of ablation experiments are designed to show the performance of our work. Finally, we present the visualization of different scenarios to verify the effectiveness of our work.