A Magnetic-Inertial Sensing Fusion Information-Driven Digital Twin Approach for the Human Hand

As an important natural medium for human interaction with the external environment, the hand has a high degree of dexterity and complexity, and accurate human hand digital twin models play an important role in the field of medical rehabilitation. In this article, magneticinertial sensing fusion information is used as driving data to build a personalized digital twin model of the human hand. A parametric hand model based on the individual phalange bioconstructive characteristics is established. The fingertip position detection problem is transformed into an inverse magnetic field problem by using a passive magnetic source as a marker with structured spatial physical field to derive the personalized individual hand parametric model. The magnetic-inertial sensing fusion information helps with the dimensionality reduction of the higher-order fullposture inversion problem without individual calibration. The derived personalized hand geometry parameters were compared with medical digital photographic markers with an error of [0.38, 2.87] mm. A personalized human hand digital twin platform driven by magnetic-inertial sensing fusion information was developed based on the MATLAB Simscape Multibody module. A dynamic twin model driver based on real-time human hand motion detection information was implemented. The sphere grasp experiments show that the relative fitting error (%) according to the derived fingertip position for the radius of 50, 60, and 70 mm is 0.14%, 0.25%, and 1.08%, respectively, demonstrating the good performance of the proposed system.