Using musculoskeletal models to generate physically-consistent data for 3D human pose, kinematic, dynamic, and muscle estimation

Human motion capture technology is utilized in many industries, including entertainment, sports, medicine, augmented reality, virtual reality, and robotics. However, motion capture data only allows the user to analyze human movement at a kinematic level. In order to study the corresponding dynamics and muscle properties, additional sensors such as force plates and electromyography sensors are needed to collect the relevant data. Collecting, processing, and synchronizing data from multiple sources could be laborious and time-consuming. This study proposes a method to generate the dynamics and muscle properties of existing motion capture datasets. To do so, our method reconstructs motions via kinematics, dynamics, and muscle modeling with a musculoskeletal model consisting of 14 joints, 40 degrees of freedom, and 15 segments. Compared to current physics simulators, our method also infers muscle properties to ensure our human model is realistic. We have met International Society of Biomechanics standards for all terminologies and representations. Furthermore, our integrated musculoskeletal model allows the user to preselect various anthropometric features of the human performing the motion, such as height, mass, level of athleticism, handedness, and skin temperature, which are often infeasible to estimate from monocular videos without appropriate annotations. We apply our method on the Human3.6M dataset and show that our reconstructed motion is kinematically similar to the ground truth markers while being dynamically plausible when compared to experimental data found in literature. The generated data (Human3.6M+) is available for download.