Generalization of inverse kinematics frameworks based on deep learning to new motor tasks and markersets

Abstract

Several systems propose to monitor the activity of workers in industry, with markerless Human Pose Estimation (HPE) methods based on deep learning. However, these systems simply provide sparse 3D human body keypoints, including noise and missing information. Hence, these sparse and noisy keypoints cannot be directly used to assess the biomechanical constraints associated with professional activity. Indeed, computing these constraints would require more accurate and high frequency motion capture data to compute reliable joint angles, or using inverse kinematics frameworks (such as OpenSim). Deep-learning (DL) based approaches, such as Opencap, have been introduced to estimate additional anatomical markers’ positions, to overcome this limitation. However, such DL-based methods rely on training datasets and predefined keypoints and markersets, and their ability to generalize to other tasks or experimental conditions is still unclear. In this paper, we assess the ability of Opencap, pre-trained with bipedal locomotion dataset, to generalize (i.e. estimate reliable 3D positions of additional anatomical markers) to bi-manual manipulation and picking tasks, and new markersets. Fine tuning, commonly used in DL to generalize a model to new data, is a promising mean to deal with unseen motions and different experimental conditions, with a few set of new training data. We evaluated the performance of various fine tuning strategies, such as retraining the full model, only the last layers or adding an additional output layer. Our results showed an important decrease of the estimation error when using fine tuning on picking and manipulation tasks, with new markersets, compared to directly applying the pretrained Opencap model. This decrease of error is obtained with a limited training dataset of 140,000 poses, which is promising for future use in new measurement conditions and unseen motions, as frequently observed in industry.

Relevance to industry: Accurate Human Pose Estimation on-site is a key challenge to accurately assess musculoskeletal disorders with relevant and reliable biomechanical variables. However, RGB-based HPE used on-site generally provide sparse and noisy postural information, which is not compatible with standard biomechanical frameworks. This paper suggests and evaluates guidelines to overcome this limitation, and to make standard HPE methods be used in biomechanical framework. This open new avenues in estimating biomechanical variables that could improve the estimation of the musculoskeletal disorders risks directly in industrial context, as it is performed in laboratory conditions. This paper could be viewed as recommendations for companies which develop ergonomic assessment tools usable in industrial context.