Advancing biomedical engineering through a multi-modal sensor fusion system for enhanced physical training

Abstract

In this paper, we introduce a multi-modal sensor fusion system designed for biomedical engineering, specifically geared toward optimizing physical training by collecting detailed body movement data. This system employs inertial measurement units, flex sensors, electromyography sensors, and Microsoft's Kinect V2 to generate an in-depth analysis of an individual's physical performance. We incorporate a gated recurrent unit- recurrent neural network algorithm to achieve highly accurate body and hand motion estimation, thus surpassing the performance of traditional machine learning algorithms in terms of accuracy, precision, recall, and F1 score. The system's integration with the PICO 4 VR environment creates a rich, interactive experience for physical training. Unlike conventional motion capture systems, our sensor fusion system is not limited to a fixed workspace, allowing users to engage in exercise within a flexible, free-form environment.