3D hand motion and position estimation using ultrasonic receiver array and inertial sensors

In this study, we introduce a novel technique for hand pose and position estimation using ultrasonic and inertial sensors. The three-dimensional (3D) position of the hand is localized by a combination of ultrasonic-based position estimation and inertial measurement unit (IMU) sensor-based 3D acceleration and pose estimation. The proposed method roughly estimates the 3D position of a hand at a relatively low speed (10 Hz) using the delay-and-sum beamforming (DSBF) algorithm by placing two multi-channel ultrasonic sensors in a two-dimensional plane; it accurately estimates the real-time 3D position using a high-speed (187.5 Hz) inertial sensor mounted on the back of the hand. To maximize the real-time performance and cost-effectiveness of the proposed system, a cost-effective field-programmable gate array is used for the ultrasonic array-based 3D localization estimation module to optimize the DSBF algorithm through parallel operations. In addition, the motions of all joints of the human hand are estimated by the attitude and heading reference system algorithm using 12 IMU sensors. Because of the high accuracy and real-time performance of the hand pose estimation method, it is expected that the proposed system will be applied to interactive virtual reality tools.